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ABSTRACT. The Kazhdan-Lusztig polynomials for finite Weyl groups arise in the geometry of Schubert 
varieties and representation theory. It was proved very soon after their introduction that they have nonneg- 
ative integer coefficients, but no simple all positive interpretation for them is known in general. Deodhar 
[Deo90] has given a framework for computing the Kazhdan-Lusztig polynomials which generally involves 
recursion. We define embedded factor pattern avoidance for general Coxeter groups and use it to character- 
ize when Deodhar's algorithm yields a simple combinatorial formula for the Kazhdan-Lusztig polynomials 
of finite Weyl groups. Equivalently, if (W, S) is a Coxeter system for a finite Weyl group, we classify 
the elements w £ W for which the Kazhdan-Lusztig basis element C' w can be written as a monomial of 
C' s where s € S. This work generalizes results of Billey- Warrington [BW01] that identified the Deod- 
har elements in type A as 321-hexagon-avoiding permutations, and Fan-Green [FG97] that identified the 
fully-tight Coxeter groups. 



1. Introduction 

The Kazhdan-Lusztig polynomials for finite Weyl groups [KL79] arise as Poincare polynomials for 
intersection cohomology of Schubert varieties [KL80] and as a (/-analogue of the multiplicities for Verma 
modules [BB81, BK81]. They are defined to be the coefficients in the transition matrix for expanding 
the Kazhdan-Lusztig basis elements in the Hecke algebra associated to the Weyl group into the standard 
basis. Several algorithms exist, formulas for special cases, and interesting properties are known for these 
polynomials; see for example [Hum90, Deo94, MW03, Pol99, LS81, Bre04, BB05]. In particular, these 
polynomials have nonnegative integer coefficients but no simple all positive formula for the coefficients 
is known in general for all Coxeter groups. 

Deodhar [Deo90] proposes a combinatorial framework for determining the Kazhdan-Lusztig polyno- 
mials of an arbitrary Coxeter group. The algorithm he describes is shown to work for all Coxeter groups 
where the Kazhdan-Lusztig polynomials are known to have nonnegative integer coefficients which in- 
cludes Weyl groups and the Coxeter groups associated to crystallographic Kac-Moody groups. Under 
certain conditions, Deodhar's algorithm for determining the Kazhdan-Lusztig polynomials turns out to 
be a beautiful combinatorial formula. These conditions are also equivalent to the Kazhdan-Lusztig basis 
element C' w being equal to a product of C' s 's indexed by generators of the Coxeter group. We say that w 
is Deodhar when it satisfies these conditions. In 1999, Billey and Warrington [BW01] gave an efficient 
characterization of the Deodhar elements in the symmetric group as 321 -hexagon avoiding permuta- 
tions. Their results extend to finite linear Weyl groups, types A, B, F, G. Our goal is to give a similar 
characterization for all finite Weyl groups. 

In this paper we give two characterizations of the Deodhar elements for all finite Weyl groups. One 
characterization is given in terms of 1-line pattern avoidance in analogy with the type A result. This 
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characterization gives a polynomial time algorithm to test for the Deodhar status of an element, but 
involves a long list of patterns. The second characterization is in terms of a new type of pattern called an 
embedded factor. These patterns are defined in terms of reduced expressions, and generalize containment 
in the 2-sided weak Bruhat order. Theorem 5. 12 states that the Deodhar elements of Weyl groups can 
be characterized by avoiding embedded factors from the following list, as well as an additional 1-line 
pattern for type D. 



Type 


Coxeter Graph 


Embedded Factor Patterns 


h{m), 
m > 3 


m 


s\S2S\, S2S1S2 (short braids) 


A 7 


•l *2 - 3 *4 *5 - 6 *7 


S5S6S7S3S4S5S6S2S3S4S5S1S2S3 {HEX) 


B 7 / Cj 


4 

•0 *1 *2 *3 *4 *5 *6 




D 6 


•l 

\ 

•l »2 »3 »4 »5 


S3S4S5S1S2S3S4S1S2S3S1 (HEX4) 


D 7 


•l 

\ 

•l »2 »3 *4 *5 '6 


S3S4S5S6S2S3S4S5S1S2S3S4S1S2S3 (HEX 2 ) 
S4S5S6S2S3S4S5S1S2S3S4SJS2S1 (HEX 3a ) 


D s 


•l 

\ 

•l »2 »3 *4 *5 '6 '7 


S4S 5 S 6 S 7 S 3 S4S 5 S e S2S 3 S4S 5 SlSlS2S 3 S4 

(diamond, to be avoided as a 1-line pattern) 


E 6 


•5 

•0 - 1 *2 # 3 # 4 


S0S1S2S5S3S4S2S3S1S2S5S0S1 
S5S1S2S3S0S1S2S5S4S3S2S1S0 
S1S2S5S3S4S2S3S1S2S5S0S1S2 
S2S5S1S2S3S0S1S2S5S4S3S2S1 


E 7 


•5 

•0 •! *2 *3 *A - 5 


S0SlS2S 3 S4S 6 S 5 S2S 3 S4SlS2S 3 S Sl 
S3S4S6S1S2S3S0S1S2S5S4S3S2S1S0 
SlS 2 S3S4S6S5S2S3S4SlS2S3S0'SlS2 
S2S3S4S6SiS 2 S 3 S SiS2S 5 S4S 3 S2Si 
S5S2S3S4SqSiS2S 5 S3S4S2S 3 S SiS2S 5 



Figure 1 . Minimal non-Deodhar patterns 



The embedded factor patterns take into account different ways of embedding one Weyl group into 
another as a parabolic subgroup. For example, the Weyl group of type Eg has parabolic subgroups of 
types A2, A 7 , Dq, D 7 , Eq, and E 7 from this list. Therefore, a Deodhar element in the Weyl group of type 
i?8 cannot have any embedded factors in the form of a short braid, hexagon, HEXi, or any of the Eq or 
E 7 patterns. 

In type D n , the Deodhar elements must also avoid the "diamond" pattern [16785234] as a 1-line 
pattern. This single 1-line pattern encapsulates an infinite antichain of type D embedded factor patterns 
further discussed in Example 5.11. 
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We also provide a finite test in Theorem 11.5 to determine when it is possible to translate between 
classical 1-line pattern avoidance and the embedded factor pattern avoidance of Definition 5.5. This result 
generalizes a fact which is implicit in [BW01] that avoiding the hexagon embedded factor pattern can 
be characterized by avoiding 4 classical permutation patterns when we restrict to the fully commutative 
elements. Theorem 11.5 also justifies using the methods in this paper to study certain classical pattern 
classes, and has been extended for type A in [Jon07]. 

In Section 2, we recall the basic definitions of Kazhdan-Lusztig polynomials and basis elements. We 
define the Deodhar elements and recall the theorem that inspired this name. In Section 3, we recall 
the heap of a reduced expression for Weyl groups of types A,B,D, and in Section 4 we review the 
definition of classical pattern avoidance. The heaps will be the main tool for proving the characterization 
in Theorem 5.12 presented in Section 5. In Section 6 we reduce the proof of the main theorem to 
short braid avoiding elements. In Sections 7 and 8, we define the convex elements and give a complete 
classification of convex Deodhar elements. Then in Section 9, we characterize the non-convex Deodhar 
elements in type D which completes the proof for type D. In Section 10, we complete the classification 
of Deodhar elements for the remaining finite Weyl groups. Section 1 1 gives a pattern comparison result, 
which shows that the Deodhar property for type D can be characterized by avoiding finitely many 1-line 
patterns. Finally, we close with some open problems and enumerative data in Section 12. 



2. Background and notation 

In this section we will set up our notation and review some of the motivation for our main theo- 
rems. For a reader unfamiliar with Coxeter groups, we recommend either the classic text by Humphreys 
[Hum90] or the recent text by Bjorner and Brenti [BB05] for a more combinatorial treatment. 

Let W be a Coxeter group with generating set S and relations of the form (siSj) m ^' = 1. The 
Coxeter graph for W is the graph on the generating set S with edges connecting and Sj labeled m(i, j) 
for all pairs i,j with m(i,j) > 2. For example, the table in Figure 1 shows the Coxeter graphs for the 
finite Weyl groups that contain minimal non-Deodhar patterns. Note that if m(i,j) = 3 it is customary 
to leave the corresponding edge unlabeled. 

An expression is any product of generators from S. The length l(w) of an element w G W is the 
minimum length of any expression for the element w. Such a minimum length expression is called 
reduced. Each element w G W can have several different reduced expressions that represent it. Given 
w G W, we represent a reduced expressions for w in sans serif font, say w = W1W2 • • • w p where each 
Wj G S. 

It is a theorem of Tits [Tit69] that every reduced expression for an element w of a Coxeter group can 
be obtained from any other by applying a sequence of braid moves of the form 



m(i,j) m(i,j) 

where Sj and Sj are generators in S that appear in the reduced expression for w, and each factor in the 
move has m(i,j) letters. Let the support of an element w G W, denoted supp(w), be the set of all 
generators appearing in any reduced expression for w, which is well-defined by Tits' theorem. We say 
that the element w is connected if supp(w) is connected in the Coxeter graph of W. 

We define an equivalence relation on the set of reduced expressions for a fixed Coxeter element where 
two reduced expressions are in the same commutativity class if one can be obtained from the other by 
commuting moves of the form SiSj i— ► SjSi, where m(i, j) = 2. In particular, if every reduced expression 
for w can be obtained from any other by commuting moves then we say w is fully commutative. By 
Tits' theorem, an element w is fully commutative if and only if no reduced expression for w contains a 
consecutive subexpression of the form SiSjSiSj ■ ■ ■ of length m(i,j) > 3. 
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We call any expression of the form SiSjSi for m(i,j) > 3 a short braid. This name reflects the fact 
that we are not considering any longer braid, even if m(i,j) > 3. We caution the reader that some 
authors have used the term short braid to refer to a commuting move between two entries Sj and sj 
where m(i,j) = 2. The short braid avoiding elements of a Coxeter group are those with no reduced 
expression containing a factor SiSjSi where Sj and Sj are any pair of generators that do not commute (i.e. 
m(i,j) ^ 2). Hence, a short braid avoiding element is also fully commutative. 

Given a Coxeter group W, we can form the Hecke algebra Ti over Z,[q 1 / 2 , q" 1 ^ 2 ] with basis {T w : 
w 6 W} and relations: 

(2.1) T S T W =T SW for l{sw) > l(w) 

(2.2) (T s ) 2 ={q - \)T S + qT x 

where T\ corresponds to the identity element. In particular, this implies that 

T — T T T 

whenever W1W2 ... w p is a reduced expression for w. 

Kazhdan and Lusztig [KL79] described another basis for H that is invariant under the Hecke algebra 
involution mapping 

q i ^ q" 1 

T s ^ (Ts)" 1 . 

This basis, denoted {C' w : w G W}, has important applications in representation theory and algebraic 
geometry [KL79, KL80]. The Kazhdan-Lusztig polynomials P X:W (q) arise as the "change of basis" 
matrix between these two bases of H: 

The C' w are defined uniquely to be the Hecke algebra elements that are invariant under the involution and 
have expansion coefficients as above where P x>w is a polynomial in q with 

, x (l(w) - l(x) - 1) 

(2.3) degree P XjW (q) < yy ' ±-± >- 

and P W)W (q) = 1. We use the notation C' w to be consistent with the literature because there is already a 
related basis denoted C w . 

For w G W and s £ S with l(sw) > l(w), the Kazhdan-Lusztig basis elements multiply according 
to the rule 

c' s c' w = c sw + ^ w ) C 'z 

sz<z<w 

where fj,(z, w) is the coefficient of q2^ w ">~ 1 ^^ 1 ) (the term of highest possible degree) in the Kazhdan- 
Lusztig polynomial P ZjW (q). This is appreciably more complicated than the corresponding multiplication 
formula (2.1) in the {T w } basis. 

Deodhar [Deo90] studied the case when C' w can be written simply as a product of C' s . 's. In this case, 
he also gives nice combinatorial formulas for all the polynomials P XtW (q). We will describe Deodhar's 
defect statistic and his theorem in terms of masks on reduced expressions. 

Fix a reduced expression w = wi • • • w^. Define a mask a associated to the reduced expression w 
to be any binary word o\ ■ • ■ of length k = l(w). Every mask corresponds to a subexpression of w 
defined by w' 7 = w^ 1 • • • where 

cr, J Wo if Oj = 1 

w„- = < 
3 [id if <7j = 0. 
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Each w CT is a product of generators in a subsequence of W! • • • so it determines an element of W that 
is less than w in Bruhat order. For 1 < j < k, we also consider initial sequences of masks denoted 
0"b1 = a i ' ' ' CT i> an d tne corresponding initial subexpressions w ^' = w^ 1 • • • wJ J . For example, we 
have w°"M = w ff . A mask a is proper if it has at least one zero. 

We say that a position j (for 2 < j < A:) of w is a defect with respect to the mask a if 

(2.4) ^w^'-^Wj) < l(w^-^). 

Note that a defect occurs in position j if w^- 7 " 1 ' satisfies the length condition above; the value of cr, is 
irrelevant when determining if j is a defect. Let d w (a) denote the number of defects of w for a mask a. 
We will use the notation d(a) = d w (f) when the reduced word w is fixed. 
We are now in a position to define Deodhar's condition. 

Definition 2.1. Let w € W be a Coxeter element with reduced expression w, and let a be a proper mask 
for w. We say that a position j is a zero-defect if o~j = and j is also a defect in w. We say that position 
j in w is a plain-zero if aj = and j is not a defect in w. Then, the mask a is Deodhar if 

(2.5) # of zero-defects of a < # of plain-zeros of a. 

Moreover, a reduced expression w is Deodhar if every proper mask a on w is Deodhar. 

Example 2.2. Assume S2 s iS3 s 2 is a reduced expression in a Coxeter group. The word/mask pair 

\s 2 Sl s 3 s 2 
W= [l 

has a zero-defect in position 4 and plain-zeros in positions 2 and 3. One can verify that (2.5) holds for 
all proper masks on w, so w is Deodhar. 

For certain Coxeter groups, Deodhar has shown that when a reduced expression w € W satisfies 
this condition, the Kazhdan-Lusztig polynomials P X;W can be obtained as the generating function that 
counts masks on w with respect to the defect statistic. Equivalently, C' w can be written as a product of 
C^.'s. He also shows that the notion of being Deodhar is well-defined on Coxeter group elements and is 
independent of the choice of reduced word used to verify the condition. Note that Deodhar actually used 
a slightly different condition which is equivalent to the one given here in (2.5) [BW01, Lemma 2]. The 
original condition is 

d(<r)<l(lM-l{W)-l) 

which comes directly from the maximum degree bound of the Kazhdan-Lusztig polynomial P w <r w (g) in 
(2.3). 

Theorem 2.3. [Deo90] Let W be any Coxeter group where the Kazhdan-Lusztig polynomials are known 
to have nonnegative coefficients, and let w = wi • • • be a reduced expression for some w 6 W. Then 
the following are equivalent: 

(1) The expression w is Deodhar. 

(2) The element w is Deodhar. 

(3) The Kazhdan-Lusztig basis element C' w is given by 

where the sum is over all masks a on w. 

(4) For all x E W, the Kazhdan-Lusztig polynomial P X)W is given by 

^(s) = E^ (CT) 

where the sum is over all masks a on w such that w a = x. 
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(5) The Kazhdan-Lusztig basis element C' w satisfies C' w = C' VJl ■ ■ ■ C' Wk . 

(6) The Bott—Samelson resolution of the corresponding Schubert variety X w is small. 

(7) The Poincare polynomial for the full intersection cohomology group of the Schubert variety X w 
is 

dim(TW 2i (X w )y = (1 + 

i 

Remark 2.4. The equivalence of (1) through (6) are implicit in Deodhar [Deo90]. The equivalence 
of (4) and (7) is proved explicitly by Billey and Warrington [BW01]. Lusztig [Lus93] and Fan and 
Green [FG97] have studied those elements for which (5) holds. These elements are called "tight" in the 
terminology of those papers. 

The main goal of this paper is to give an efficient way to identify Deodhar elements. In the case when 
W is the symmetric group, Billey and Warrington [BW01] gave a concise description of the Deodhar 
elements as those that are 321-hexagon avoiding, where the term "hexagon" comes from the notion of 
a heap on a permutation. We will describe the heap construction and classical pattern avoidance in the 
next two sections and return to the study of Deodhar elements in Section 5. 

3. Heaps and string diagrams 

Each reduced expression can be associated with a partial order called the heap that we define below. 
This partial order allows us to visualize a reduced expression as a set of lattice points while maintaining 
the pertinent information about the relations among the generators. Carrier and Foata [CF69] were among 
the first to study heaps of dimers, and these were generalized to other settings by Viennot [Vie89]. More 
recently, Stembridge has studied enumerative aspects of heaps [Ste96, Ste98] in the context of fully 
commutative elements. We will use the heaps to visualize the reduced expressions that appear in the 
table on Page 2 and prove our characterization in type D. 

Definition 3.1. Suppose w = wi • • • is a fixed reduced expression, and define a partial ordering on 
the indices {1, • • • , k} by the transitive closure of the relation i < j if i < j and m(wj, Wj) ^ 2. In 
particular, i < j if i < j and = wj. This partial order is called the heap of w. We label the element i 
of the poset by the corresponding generator Wj. 

Remark 3.2. Observe that heaps are well defined up to commutativity class, so if u and v are two reduced 
expressions for w in the same commutativity class then the labeled heaps of u and v are equal. In partic- 
ular, if w is fully commutative then there is a unique labeled heap poset for the element w, regardless of 
which reduced expression is used to generate it. 

Let G denote the Coxeter graph for W. We can embed the heap poset as a set of lattice points in 
GxN. To do this, begin by reading the reduced expression w from left to right, and drop a point in 
the column representing each generator Wj. We can envision each point as being "fat" and under the 
influence of "gravity," in the sense that the point must fall to the lowest possible position in the column 
over the generator corresponding to w, in the Coxeter graph without passing any previously placed points 
in adjacent columns. Here, we say two columns are adjacent when they correspond to adjacent vertices 
in the Coxeter graph. Since generators that are adjacent in the Coxeter graph do not commute, we must 
place the point representing Wj at a level that is above the level of any other adjacent points that have 
already been placed. Because generators that are not adjacent in the Coxeter graph do commute, points 
that lie in non-adjacent columns can slide past each other or land at the same level. 

Definition 3.3. Let w be a reduced expression for a Coxeter element. We let Heap(w) denote the lattice 
representation of the heap poset in G x N constructed as described in the preceding paragraph. (We will 
amend this definition in Example 3.7 to account for the fork in the Coxeter graph of type D.) 
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We will give several examples of heaps in different Coxeter groups in the next three examples and 
also introduce some useful terminology for permutations and signed permutations. 

Example 3.4. The Coxeter graph of type A n ^\ is the following: 

•l »2 »3 ' ' ' *n-l . 

The corresponding Coxeter group is the symmetric group S n . We may refer to elements in the symmetric 
group by the 1-line notation w = [w\W2W^ ■ ■ ■ w n ] where w is the bijection mapping i to u?j written in 
italic font. The generators s\, S2, ■ ■ ■ , s n ~i are the adjacent transpositions where Sj interchanges i and 
i + 1. For example, when multiplying a permutation on the right by S2, we interchange the entries in 
positions 2 and 3 of the 1-line notation, so for w = [3412] € 54 we have w-S2 = [3142]. Dually, when 
multiplying on the left by S2, we interchange the digits 2 and 3 in the 1-line notation for the element, so 
S2W = [2413]. One reduced expression for w is S2 s 3 s l s 2- We build up the heap one generator at a time 
for the reduced expression w = S2S$s\S2 in type A3 as shown below. 



• • • • • 

123 123 123 123 

We can view the points in the lattice as the vertices in the Hasse diagram for the heap poset where the 
edges are implied by the Coxeter graph. Note that the other reduced expression S2S1S3S2 for [3412] 
corresponds to a different linear extension of the heap above. 

Remark 3.5. In the lattice representation of a heap poset, all of the entries of the reduced expression that 
correspond to the same generator lie in a column over the given generator in the Coxeter graph. Each 
entry will have a certain level in the heap, but the poset is not ranked. In the example below, the reduced 
expression w = S1S4S2S3 has a heap where the rank of the S3 entry is not well defined and the level of 
the S4 entry is an artifact of the way we imposed "gravity" in the construction. 



12 3 4 

In Section 7, we will further refine the lattice representation of the heap by coalescing the connected 
components so entries that are connected in G x N satisfy the covering relation in the heap poset. 



In type A, the heap construction can be combined with another combinatorial model for permutations 
in which the entries from the 1-line notation are represented by strings. The points where two strings 
cross can be viewed as adjacent transpositions of the 1-line notation. Hence, we can overlay strings on 
top of a heap diagram to recover the 1-line notation for the element, by drawing the strings from bottom 
to top so that they cross at each entry in the heap where they meet and bounce at each lattice point not 
in the heap. Conversely, each permutation string diagram corresponds with a heap by taking all of the 
points where the strings cross as the "fat" points of the heap and letting them "fall" according to the 
relations given by the Coxeter graph. 

For example, we can overlay strings on the two heaps of [3214]. Note that the labels in the picture 
below refer to the strings, not the generators. 
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3 2 1 4 3 2 1 4 

1 2 3 4 1 2 3 4 

The string diagram helps us to visualize the relationship between the 1-line notation for a permutation 
and the corresponding heap. Fixing a reduced expression w for a permutation, there exists a string 
diagram that is obtained from Heap(w) by adding strings that cross at each lattice point of Heap(w). 
Observe that we are able to read off the 1-line notation for an element by labeling the strings 1, . . . , n 
along the bottom and then reading the corresponding labels from the top. We can also obtain reduced 
expressions from any string diagram by reading the string crossings as generators in any order that is 
consistent with the implied heap poset structure. 

Example 3.6. The Coxeter graph of type B n is of the form 

4 

•0 *l *2 *3 ■ • • — •n-1 ■ 

From this graph, we see that the symmetric group S n is a parabolic subgroup of this Coxeter group. That 
is, S n is generated by a subset of the generators of B n . Because of this, the elements of this group have a 
standard 1-line notation in which a subset of the entries are barred. We often think of the barred entries 
as negative numbers, and this group is referred to as the group of signed permutations or the hyperocta- 
hedral group. The action of the generators on the 1-line notation is the same for {s±, S2, ■ ■ ■ , Sn-i} as in 
type A in which wsi interchanges the entries in positions i and i + 1 in the 1-line notation for w. The so 
generator acts on the right of w by changing the sign of the first entry in the 1-line notation for w. For 
example, w = [4231] is an element of B4 and 

ws = [4231] 
ws! = [2431]. 

Note that because the edge in the B n Coxeter graph connecting sq and s\ is labeled 4, we have that 
sosisosi and S1S0S1S0 we reduced expressions for the same element denoted [1234 ... n] in 1-line no- 
tation. 

The heap for a type B reduced expression will look like the heap of a type A expression because its 
Coxeter graph is a path. As in type A, we can adorn these heap constructions with strings that represent 
the digits of the 1-line notation for the element. If we label the strings at the bottom of the diagram 
with the numbers from 1, . . . , n then the so generator has the effect of bouncing the string back in the 
direction from which it came, while changing the sign of the label for the string. All other generators 
cross the strings as in type A. 

Example 3.7. The Coxeter graph for type D n is shown below. 

•1 

\ 

•l *2 *3 *4 ... — •n-1 

The elements of type D can be viewed as the subgroup of B n consisting of signed permutations with 
an even number of barred entries. The action of the generators on the 1-line notation is the same for 
{si, S2, ■ ■ ■ } as in type A in which ws,i interchanges the entries in positions i and i + 1 in the 1-line 
notation for w. The generator acts on the right of w by marking the first two entries in the 1-line 
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notation for w with bars and interchanging them. For example, v = [4231] and w = [4231] are elements 
of D4 and 

vsi =[2431] 
wsi =[2431]. 

Although the Coxeter graph for type D has a fork, we will draw the heap of type D elements in a 
linearized way by allowing entries in the first column to consist of either generator or both: 

s\ =• 
si =• 

S\Sl = ••. 

We denote this linearized lattice point representation by iJeap(w). Hence, if w € D n then Heap(\N) is 
a subset of [n — 1] X N rather than GxN, where [n — 1] = {1, 2, . . . , n — 1}. 

As in type A, we can adorn Heap(w) with strings that represent the digits of the 1-line notation for 
the element. If we label the strings at the bottom of the diagram with the numbers from 1, . . . , n, then the 
sj generator crosses the strings that intersect it and changes the sign on the labels for both strings. All 
other generators simply cross the strings as in type A. For example, the heap of the reduced expression 
SJS2S3S1S2SJS1 = [3421] is given below. 



3 4 2 1 




si s 2 S3 



4. Classical pattern avoidance 

The 1-line notations for types A,B,D carry a notion of pattern containment that generalizes the 
following classical definition. 

Definition 4.1. Let w = [w\ . . . w n ] be a permutation in S n written in 1-line notation as described in 
Example 3.4. Let p = [pi ■ ■ ■ Pk] be another permutation in Sk for k < n. Then we say w contains the 
permutation pattern p if there exists a subsequence 1 < i\ < 12 < . . . < ik < n such that 

w ia < w i b Pa < Pb 

for all 1 < a < b < k. If w does not contain p then we say that w avoids the permutation pattern p. 

In other words, w contains p if there exist k rows and columns in the permutation matrix for w 
whose common entries are the permutation matrix for p. For example, w = [53241] contains the pattern 
p = [321] in several ways including the underlined subsequence, and w avoids q = [1234]. 

One of the earliest uses of permutation patterns occurred in computer science [Tar72]. A good in- 
troduction to enumerative methods in pattern avoidance can be found in [Bdn04]. Several interesting 
properties of Schubert varieties, Kazhdan-Lusztig polynomials and Bruhat order can be characterized 
by pattern avoidance [LS90, BP05, BB03, Gas98, WY06b, WY06a, Ten06a, BMB06]. 

A property of permutations can be characterized by pattern avoidance if there is no w having the 
property and p not having the property such that w contains pas a pattern. Equivalently, the subset of 
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permutations that do not have the property must be an upper order ideal in the poset on = [j n>0 S n 
ordered by pattern containment. The property is then characterized by avoiding the minimal elements of 
this upper order ideal. 

For example, the property of a permutation being short braid avoiding in type A is characterized by 
avoiding the pattern [321], first noted by [BJS93]. Hence, the fully commutative elements in S n are 
enumerated by the Catalan numbers [SS85]. Also, checking a permutation of length n for a subsequence 
of length 3 can be done in 0(n 3 ) time while looking at all reduced words for a typical permutation takes 
an exponential amount of time in n. A pattern avoidance characterization is most useful when the set of 
minimal patterns is finite, but this need not be the case [SB00]. 

When w = \w\ . . . w n ] is a signed permutation from type B or type D, then we say that w contains 
the 1-line pattern p = [p\ . . .pt] if the underlying permutation for w, obtained by ignoring the bars in 
the 1-line notation, contains the underlying permutation for p, and assuming the pattern instance occurs 
in positions i\ < i% < ■ ■ ■ < i^, we further require that wi is barred only if pj is barred. For example, 
the type D element w = [53241] contains the pattern p = [321] as the underlined subsequence [53241], 
but not the underlined subsequence [53241] because the pattern of bars does not match. 

Some early applications of pattern avoidance in types B and D occurred in [Bil98, BL98]. Type B 
enumerative results have been obtained by [Bec97, SimOO, MW04], and there are also some extensions to 
colored permutations [Man03, Man02]. Classical pattern avoidance extends to all Coxeter groups using 
the notion of root subsystems described in [BP05, BB03]. 



Let W be an arbitrary Coxeter group with w,y G W. We say that w contains y as a factor if there 
exist elements a and b in W such that w = ayb and l(w) = 1(a) + l(y) + 1(b). Equivalently, w contains 
y as a factor if some reduced expression for w contains some reduced expression for y as a consecutive 
subword, i.e. w = wiW2...w p and y = WjW/ + i • • • w j for some 1 < i < j < p. The induced partial 
order on Coxeter group elements is known as the two-sided weak Bruhat order [BB05]. We will use 
the following lemma to show that the Deodhar elements form a lower order ideal in the two-sided weak 
Bruhat order. 

Lemma 5.1. Let W be a Coxeter group. If w € W is Deodhar then w~ l is Deodhar. 
Proof. Let w = wi . . . w p be a reduced expression for w, and consider the two products 



5. Deodhar elements of Coxeter groups 



(5.1) 




(5.2) 




It follows directly from the symmetry in the multiplication rule (2.1) that 

Ru(q) = S u -i(q). 
By Deodhar's Theorem 2.3, w is Deodhar if and only if 




for all u and S w -i(q) = 1 
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by definition of C' w _ 1 since the product of C^.'s is invariant under the involution. Therefore, w 1 is 
Deodhar. □ 

The following proposition can easily be derived from Proposition 2.1.4 of [FG97]. We include an 
independent proof for completeness. 

Proposition 5.2. Let w,y G W be Coxeter group elements. If y is not Deodhar and w contains y as a 
factor, then w is not Deodhar either. 

Proof. Suppose y is not Deodhar and y = yi • • • y p is a reduced expression. Then by (2.5) there exists a 
proper mask a for yi • • • y p with 

# of zero-defects of a > # of plain-zeros of a. 

Consider multiplying y on the right by a generator s 6 S such that l{ys) > l{y). We can extend the mask 
a by placing a 1 in the last position to obtain a mask for yi • • • y p s. Since the inequality above remains 
unchanged for the new mask, the element ys is not Deodhar. 

Next, consider multiplying y on the left by a generator s G S such that l(sy) > l(y) or equivalently 
Kv s ) > Kv )• By Lemma 5.1, y is not Deodhar implies y" 1 is not Deodhar. So by the argument 
above, y~ 1 s is not Deodhar and so neither is sy. 

The theorem follows by induction on l(w) — l(y). □ 

We immediately obtain a combinatorial proof of a result from [FG97] which shows that the Deodhar 
elements are all short braid avoiding. 

Corollary 5.3. Let w G W be a Coxeter group element. If w contains a short braid then w is not 
Deodhar. 

Proof. Say Sj, Sj are noncommuting generators. The reduced expression/mask pail- 
s' Sj S{ 

1 

has a zero-defect in the last position hence the number of zero-defects equals the number of plain zeros. 
Consequently, SiSjSi is not Deodhar. Therefore, any element that contains a short braid as a factor in a 
reduced expression is not Deodhar. □ 

Proposition 5.2 also shows that the non-Deodhar elements form an upper order ideal in the two-sided 
weak Bruhat order. In order to obtain an efficient generating set for this ideal, we consider a refinement 
of factor containment. 

Definition 5.4. Let W, W be Coxeter groups with associated Coxeter graphs G, G' respectively. Then, 
a Coxeter embedding of G' is an injective map of the generators f : G' —> G that restricts to a labeled 
graph isomorphism onto its image. 

A Coxeter embedding induces an injection of W' into W, and we will abuse notation and call this 
map / : W' — > W a Coxeter embedding also. When phrased algebraically, a Coxeter embedding / : 
W' — > W is an injective map of generators for the Coxeter group such that m(sj, Sj) = m(f(si), f{sj)) 
for all Si, Sj G W. Since / is a map of generators, we can extend it to a map of Coxeter group elements 
by treating it as a word homomorphism on any reduced expression in W'. 

Definition 5.5. Suppose W is a Coxeter group, and w,y G W. Let W y be the parabolic subgroup whose 
generators are determined by the support of y. If there exists a Coxeter embedding / : W y — ► W such 
that w contains f(y) as a factor, then we say that w contains y as an embedded factor. 

This definition yields a stronger reformulation of Proposition 5.2, which enables us to characterize 
Deodhar elements with a shorter list of non-Deodhar patterns. 
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Corollary 5.6. Let y be a Coxeter element that is not Deodhar. If w is a Coxeter element that contains 
y as an embedded factor, then w is not Deodhar either. 

Proof. If a is a mask for a reduced expression y then it follows from Definition 5.4 that a is also a mask 
for /(y). This mask has defects in exactly the same positions as it does when it is applied to y. Hence, y 
is Deodhar if and only if f(y) is Deodhar, for any Coxeter embedding / : W y — ► W. □ 

Example 5.7. In the Coxeter group of type H4, [24513] = S1S2S3S1S0S1S2S1S0S4S3S4S1 contains the 
factor S4S3S4 in the parabolic subgroup generated by {53, S4}. This subgroup is isomorphic to S3 and 
S4S3S4 maps to S2S1S2 = [321] € S3, so [24513] contains [321] as an embedded factor. Consequently, it 
is not Deodhar. 

Example 5.8. In type A, the Coxeter embeddings of connected subgraphs are simply shifts of the gen- 
erators along the linear Coxeter graph or reversed shifts. In particular, if the generators of S& are la- 
beled si, S2, - - - , in its Coxeter graph, then the image of the generators under a Coxeter embed- 
ding / : Sfc — > S n are either of the form s\ + j,S2+j, ■ ■■ , Sfc-i+j or Sk-i+j, ■ ■ ■ , S2+jS\ + j for some 
< j < n - k. 

The hexagon avoiding elements of a Coxeter group are the ones that avoid the element 
u = S3S2S1S5S4S3S2S6S5S4S3S7S6S5 = [46718235] 
of A7 as an embedded factor. The name arises from the shape formed by the heap of u: 




Sl S 2 S3 S 4 S 5 S 6 S 7 



Billey and Warrington have given a complete characterization of the Deodhar elements in linear Weyl 
groups where the Coxeter graph consists of a single path. We can now state their theorem precisely. 

Theorem 5.9. [BW01] In types A n , B n , an element w is Deodhar if and only ifw avoids short braids 
and hexagons as embedded factors. 

Our main theorem below generalizes this theorem. It is a concise characterization of the Deodhar 
condition for the other finite Weyl groups. 

Remark 5.10. The techniques used in [BW01] do not easily extend to the remaining finite Weyl groups 
of types D and E. Among the connected short-braid avoiding heaps of type A, the notions of coalesced 
heap containment and embedded factor containment are essentially the same up to the orientation of the 
Coxeter graph. This is not the case in type D. In addition, the non-Deodhar elements of type D can have 
heaps containing "alcoves" or "holes" so that the heap lattice points do not form a laterally convex set in 
the sense of [BW01]. 

Example 5.11. In type D, there is an infinite antichain of non-Deodhar elements (i.e. no pair of elements 
from the family contain each other as embedded factors) whose heaps can contain "alcoves:" 
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FLHEXq FLHEX x FLHEX 2 

We have drawn the heaps of these elements with decorations that indicate a particular mask, as described 
in Section 6. The masks shown demonstrate that these elements are not Deodhar. 

This example shows that the set of type D Coxeter elements partially ordered by embedded factor 
containment is not well quasi-ordered. Also, there is a simple example showing that the permutations 
partially ordered by embedded factor containment is not well quasi-ordered. See [SBOO] for an analogous 
example in the classical permutation pattern poset. 

We can still obtain a finite characterization for the Deodhar condition in type D since all of the 
FLHEXx elements contain FLHEXq = [16785234] as a classical 1-line pattern, in the manner de- 
scribed in Section 4. 

Theorem 5.12. Let w £W be an element of a finite Weyl group. Then, the following are equivalent: 

(1) The element w is Deodhar. 

(2) The element w~ l is Deodhar. 

(3) The element w avoids the short list of embedded factor patterns given in Figure 1, as well as the 
FLHEXq 1-line pattern of type D. 

Proof. The equivalence of (1) and (2) follow from Lemma 5.1. The equivalence of (1) and (3) follows 
from Theorem 5.9 for types A and B, Theorem 9.2 below for type D, and Theorem 10.1 for the finite 
exceptional groups. This accounts for all irreducible finite Weyl groups. □ 

The proof will occupy Sections 6 through 10. For the finite exceptional groups, a brute-force search 
implemented on a computer suffices. Our work is simplified by the fact that we only need to check the 
short braid avoiding elements of these groups. For the infinite type D family, we need to show that our 
list of minimal non-Deodhar elements is complete. This is shown in Theorems 8.1 and 9.1. The proof of 
Theorem 8. 1 involves a map from the heap of a type D reduced expression to a type A heap where the 
Deodhar condition was already known by Theorem 5.9. We complete the classification by checking that 
this map preserves the Deodhar property. 



6. Short-braid avoiding heaps in type D 

This section develops the heap technology necessary to carry out the classification of minimal non- 
Deodhar elements in type D under embedded factor containment. For our work in this section, it suffices 
by Corollary 5.3 to consider only short braid avoiding elements. Short braid avoiding elements are fully 
commutative so they have a unique heap poset. We draw the heaps of type D elements in a linearized 
way as described in Example 3.7, with entries corresponding to both s\ and generators in the first 
column, and denote this lattice point representation of the heap by Heap(w). We will consider masks to 
be assignments of 0's and l's to the entries in the heap instead of 0,1-sequences associated to a particular 
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reduced expression. Of course, a reduced word/mask pair can be read off from the heap by reading the 
entries in order of any linear extension of the heap. 

We decorate Heap(w) according to mask-value using the following table: 



Decoration 


Mask-value 


o 


zero-defect entry 


o 


plain-zero entry (not a defect) 


• 


mask-value 1 entry 


-k 


entry of the heap with unknown mask-value 


• 


lattice point not necessarily in the heap 


* 


lattice point that is definitely not in the heap 




lattice point that is highlighted for emphasis 



In the decorated heaps, we don't distinguish one-defects from plain-ones since they don't contribute to 
the Deodhar bound (2.5). 

We can adorn our decorated heaps with strings representing the digits of the standard 1-line notation 
for type D elements as described in Section 3. Given a decorated heap, the strings will cross only at 
entries corresponding with mask- value 1. At mask- value entries the strings "bounce" as if the entry 
were not in the heap. If the decorated heap corresponds to a reduced expression w and mask a, then the 
resulting labels on the strings at the top of the heap will be the 1-line notation for w' 7 . Recall that a string 
passing through a • entry corresponds to a generator so it changes sign (which is not shown explicitly 
in our pictures). 

We can use the strings to obtain a useful test for the defect status of a particular entry in a decorated 
heap. Note that at every entry in the heap, two strings approach it from either side. We will call these the 
left string and the right string for that entry. 

Lemma 6.1. Consider an entry p in a heap. Draw the left and right strings emanating downward from 
p and label the string that ends up leftmost on the bottom by 1, and the string that ends up rightmost on 
the bottom by 2. The entry p is a defect if and only if the strings are labeled at the top according to the 
following table of patterns. 



Ifp corresponds to the generator . . . 


. . . then p is a defect if and only if the strings 
are top-labeled 


s i 


12, 12, 21 or 21 


any other generator 


12, 12, 21 or 21 



Note that when p is not sj, this test is just a signed version of the usual type A inversion. 



Proof. This follows because the length formula in type D for an even signed permutation given in one 
line notation w = [wiW2 ■ ■ ■ w n ] is l(w) = #{i < j : Wi > iUj} + #{i < j :wl > Wj}, viewing the 
barred entries as negative. □ 
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Example 6.2. The following decorated heaps with strings demonstrate the defect status of the top entry 
in the masked expressions below. 

S2 S 3 S 4 SJ S 2 S3 Si S 2 Si 

1 1 1 1 



2 1 

1 2 

1 2 

Here is an observation that is used extensively in the classification. 

Corollary 6.3. The strings for a defect in any type D heap must cross at least once strictly below the 
defect. 

Proof. If the strings for an entry never cross then the right string of the entry will be labeled 2, which 
does not match any of the defect labelings in Lemma 6. 1. □ 

Example 6.4. In type A, an entry is a defect if and only if its strings cross an odd number of times below. 
By contrast, in type D it is possible for the strings of an entry to cross below, yet not form a defect. For 
example, 



2 1 




1 2 



In this case, the strings are interchanged but both negatively signed, so the values are in increasing order. 
Hence, a is not a defect by Lemma 6.1. 

Let w € D n be a reduced expression for a connected short-braid avoiding element. Suppose x and y 
are a pair of entries in Heap(w) that correspond to the same generator Sj, so they lie in the same column 
i of the heap (setting i = 1 in case the generator is sj). Assume that x and y are a minimal pair in the 
sense that there is no other entry between them in column i. Then, for w to be reduced there must exist 
at least one non-commuting generator between x and y, and for w to be short braid avoiding there must 
actually be two non-commuting generators that lie strictly between x and y in Heap(w). We call these 
two non-commuting generators a resolution of the pair x, y. 

Definition 6.5. If both of the generators in a resolution lie in column i — 1 (i + 1, respectively), we call 
the resolution a left (right, respectively) resolution. If the generators lie in distinct columns, we call the 
resolution a distinct resolution. 



w 



S3 S 4 S 2 Si S 3 S 2 

10 10 10 



2 1 
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In type D, every resolution of a minimal pair must be one of these types. Note that the s\ and sj 
generators lie in the same column of the type D heap so although S2S1SJS2 is a fully commutative 
element in D 3 , the pair of s 2 entries do not have a distinct resolution. On the other hand, the pair of 
S2 entries in S1S2S1S3S2S1 has a distinct resolution, while the pair of si generators has only a right 
resolution. 

Recall that by Tits' theorem a permutation is fully commutative if and only if every minimal pair has 
a distinct resolution. Short braid avoiding is equivalent to fully commutative in type D. We establish 
some structural lemmas about the resolutions in Heap(w) when w is short braid avoiding. 

Lemma 6.6. Let w G D n be a short braid avoiding element. Then, no minimal pair of generators in 
column i > 2 in Heap{w) can have a right resolution. 

Proof. Since w is short braid avoiding, if x and y are resolved by a pair of generators Z\ , Z2 that lie to the 
right of column i then Z\ and Z2 necessarily correspond to the same generator since the Coxeter graph of 
type D is a path beyond column 1. Choose z\, Z2 to be a minimal pair in column i + 1. Since x and y 
form a minimal pair, we cannot backtrack when resolving z\ and Z2, so z\ and Z2 must be separated by 
another minimal pair of generators to the right that are non-commuting with z\ , z 2 : 

• x 

.21 



,2 2 

•v 

Since D n is finitely generated, eventually there exists a minimal pair of entries in the rightmost column of 
Heap(w) that cannot be resolved. Hence, every minimal pair of generators in column i > 2 is resolved 
by two generators from distinct columns or by a pair of generators to the left. □ 

Lemma 6.7. Let w 6 D n be a short braid avoiding element. 

(1) There exists an s j between every minimal pair ofs\ generators, and an s\ between every minimal 
pair of sj generators in Heap(w). 

(2) If there is an entry in which both s\ and sj lie at the same level of Heap{w), then column 1 
contains no other entries. 

(3) If there exists a pair of entries in column 1 corresponding to the same generator then all of the 
entries in the first column must be on distinct levels of the heap, and they must alternate between 
the generators s\ and sj. 

Proof. Part (1): Suppose that x and y are a minimal pair in column 1 corresponding to s±. Then they 
must be resolved by a pair z±, Z2 in column 2 since s\ commutes with every other generator. We can 
choose z\, Z2 to be a minimal pair. Since z\, Z2 cannot have a right resolution by Lemma 6.6, there exists 
an entry between z\ and Z2 in column 1. By the minimality of x, y, this entry must correspond to sj. 
Moreover, z±, Z2 cannot use a left resolution without contradicting the minimality of the pair x and y. 
Hence, Z\ , Z2 have a distinct resolution with a heap fragment of the form: 

.2 

.21 
.22 

The same argument with the roles of si and reversed shows that there is an si between every minimal 
pair of sj generators. 
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Part (2): Suppose si and sj lie at the same level k of the heap. If column 1 contains another entry p, 
suppose without loss of generality that p forms a minimal pair with the si entry from level k of the heap, 
and there are no other entries in column 1 between p and the si entry from level k. Then, in order for 
the heap to correspond with a reduced expression, this pair must have a right resolution using a minimal 
pair of s^'s. But resolution of the minimal pair of S2 generators includes an entry from column 1 by 
Lemma 6.6, contradicting the minimality of our choice of p from column 1. Thus, there cannot be other 
generators in the first column. 

Part (3) follows directly from Parts (1) and (2). □ 

Lemma 6.8. Let w G D n be a short braid avoiding element. If there exists a minimal pair of entries 
x,y in column i > 2 with only a left resolution, then the part of the heap to the left of column i has a 
particular form shown below with exactly two entries in each of columns 1, . . . , i: 



•v 

Proof. Suppose x and y are separated by a minimal pair of generators z\ , Z2 both lying in column i — 1. 
Applying reasoning similar to that in Lemma 6.6, z\ , Z2 must have a left resolution as well by minimality. 
Continuing on, we eventually obtain a minimal pair in column 2. In contrast to the case of a right 
resolution, we can resolve a minimal pair in column 2 to the left with sisj, and by minimality we must 
do so. Since these generators commute, they lie at the same level in the heap. 

Observe that by minimality, there can be no entries between x and y, nor between any of the other 
minimal pairs in the left resolution. Note also that there cannot be any other entries in column 1, by 
Lemma 6.7. Therefore, if there existed any other entries in columns 2, . . . , i above or below the mini- 
mal pairs, then they would create new minimal pairs with the existing entries and would require a left 
resolution or a distinct resolution by Lemma 6.6. But resolving these implied minimal pairs eventually 
requires additional entries in column 1, which contradicts Lemma 6.7(2). Thus, there can be no other 
elements in columns 2, . . . , i. □ 

Definition 6.9. Observe that if position k is a defect of w = wi . . . w p with respect to the mask a, then 
w CT [ fe_1 lwjt is not reduced. Hence by Tits' theorem, there must be some entry Wj with mask- value 1 that 
lies to the left of in the reduced expression and corresponds to the same generator as w^. We call the 
rightmost such position the critical generator for the defect k. 

The critical generator can be viewed as an element of the heap as well. The critical generator is always 
the first entry in the heap below k and in the same column as k. 

Lemma 6.10. Suppose w 6 D n is a connected, short braid avoiding, non-Deodhar element and H eap(w) 
contains a minimal pair of entries with only a left resolution. Then, we can construct an element w £ 
with k < n such that: 

(1) w contains w as a 1-line pattern, 

(2) every minimal pair of entries in Heap(w) has a distinct resolution, and 

(3) w is a connected short-braid avoiding non-Deodhar element. 

Proof. Suppose that the rightmost pair of entries that require a left resolution lie in column i. By 
Lemma 6.8, we have a specific form for columns 1, . . . ,i in Heap(w). To construct Heap(w) from 
Heap(w), we remove columns 2 through i, and shift columns i + 1, i + 2, . . . to the left. Hence, column 
1 of Heap(w) contains an si«j entry leftover from the left resolution, and column j > 2 of Heap{w) 
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contains the entries of column j + i — 1 of Heap{w). In the example below, i = 3 and Heap{w) is 
obtained by removing the 4 gray entries. 




By Lemma 6.8, the strings labeled 2, 3, . . . , i on the bottom of the heap end up in the same positions 
at the top of the heap. Therefore, the strings of Heap(w) can be canonically identified with strings 
1, i + 1, . . . , n, of Heap(w), so the resulting string diagram for w is well-defined with w containing w 
as a 1-line pattern. Furthermore, since w is connected, so is w. 

By the minimality assumption and Lemma 6.6, every minimal pair of entries in Heap(w) to the right 
of column i has a distinct resolution. Therefore, by construction every pair in Heap(w) has a distinct 
resolution. In particular, w is short braid avoiding. 

Finally, if w is not Deodhar then we can find a proper non-Deodhar mask a on Heap(w). We show 
this implies there exists a non-Deodhar mask on Heap(w). Recall from (2.5) that a mask is non-Deodhar 
whenever 

# zero-defects > # plain-zeros 

which we refer to as the non-Deodhar bound throughout the proof. 

Consider the effect on the non-Deodhar bound of modifying every mask value in a to be 1 in columns 
2, . . . , i. If we set a plain-zero that is not involved with any defect to have mask-value 1 then the mask 
remains non-Deodhar. 

Say there exists a zero-defect at the top of column h < i. Then the strings for the defect must cross 
by Corollary 6.3. By Lemma 6.8, the form of the heap in columns 1, . . . , i is determined. Hence, the 
left string of the defect must travel southwest from the defect until it hits a zero in column 1 < g < h, 
drop straight down until it hits the next entry in the heap which must also be a zero, and then continue 
southeast until it crosses the right string of the defect at the bottom entry in column h. Both of the entries 
in column g must have mask-value to facilitate the string crossing for the defect. Neither entry in 
column g can be a zero-defect, because every defect must have a critical generator below it in the same 
column. As we already assumed that the mask values of both entries in column g were 0, the lower entry 
is not a critical generator. Thus, we find that setting the entries in columns g and h to have mask- value 1 
removes a zero-defect and two plain-zeros, which preserves the non-Deodhar bound for the mask. 

If there is no zero-defect in any column j > i that has a string passing through column i, then the 
mask a' obtained from a by setting the mask- values of all entries in columns 1, . . . , i to 1 will remain 
non-Deodhar, since the defect status depends only on the string dynamics for the left and right strings of 
an entry by Lemma 6.1. 

On the other hand, if there exists a zero-defect in column j > i whose left string encounters column 
i, then in order for the strings of the defect to cross, the path of the left string must follow a similar 
course as described above. That is, the left string travels southwest to a plain-zero, say in column f < i, 



EMBEDDED FACTOR PATTERNS FOR DEODHAR ELEMENTS IN KAZHDAN-LUSZTIG THEORY 



19 



drops straight down to another plain-zero in the same column, and continues southeast through column 
i again and beyond. In this case, the mask a' is obtained from a by setting the entries in column 1 to 
have mask- value 0, and setting the mask- values of entries in columns 2, . . . , i to 1. This will maintain 
the string dynamics for the left string of the defect in column j, and effectively moves the plain zeros 
from column / to column 1 . Hence, the non-Deodhar bound is preserved. 

In either of the definitions for a' above, the strings 2, . . . , i cross once to the right then once to the left 
so remain in their original position. In particular, all the plain-zeros and zero-defects in a' applied to w 
remain if we remove the strings 2, . . . , i. Therefore, a' restricted to columns 1, i + 1, • • ■ , n determines 
a non-Deodhar mask for w. □ 



7. Convex elements in type D 

In this section, we restrict our attention to a subset of the short braid avoiding elements that have the 
lateral convexity property introduced for type A in [BW01]. 

Definition 7.1. If w G D n is short braid avoiding and every minimal pair of entries in Heap(w) has a 
distinct resolution then we say w is convex. 

Remark 7.2. It follows immediately from the definition that in a convex type D heap, there can only be 
a single generator in the rightmost column. 

For example, a permutation is short braid avoiding if and only if it is convex. The element S2SisjS2 € 
Z?4 is not convex since it does not have a distinct resolution of the S2's, while S2S1S3SJS2 €E D4 is convex. 

It follows from [SS85] that the number of short braid avoiding elements in type A n is the Catalan 
number c n = ^j-j- ( 2 ^ 1 ). In types B n and C n , the short braid avoiding (equivalently, convex) elements 
are also counted by Catalan numbers. In type D, Fan and Stembridge have given an explicit formula for 
the number of short braid avoiding elements [Ste98]. Furthermore, Stembridge characterized the short 
braid avoiding elements of type D in terms of 1-line patterns. 

Theorem 7.3. [Ste97] An element w in type D is short braid avoiding if and only ifw avoids all 1-line 
patterns [abc] where \a\ > b > c or b > \a\ > c 

In type D, there is a new sequence corresponding to the number of convex elements in D n for n > 1. 
The first 10 terms are 

1, 4, 13, 44, 154, 552, 2013, 7436, 27742, 104312 

Remark 7.4. This sequence is also characterized by 1-line patterns. The patterns are simply the patterns 
from Theorem 7.3 and the single additional pattern [123] € -D3. It would be interesting to know more 
about this sequence. The notion of a convex element can be extended to other Coxeter groups where the 
Coxeter graph can be linearized in a meaningful way. 

Definition 7.5. Given a convex element w € D n , we define an operation called coalescing, which 
connects the lattice point components of the heap as in [BW01]. Starting in column 1, if there exists 
an entry in column 1 below an entry in column 2 with empty lattice points between them, then allow 
the entry in column 1 to rise up until it is blocked by the entry in column 2. Then, allow all the entries 
below the column 1 entry to rise up as well, until blocked by a non-commuting generator. Work to the 
right, continuing to apply the same elevations until the heap is pushed together as much as possible. The 
resulting collection of lattice points is called the coalesced heap of w. 

Throughout the rest of the paper, whenever we refer to a position, we will mean the coordinates (a, b) 
in the lattice 1? containing a coalesced heap for a convex element. Here a is the column number and b is 



20 



SARA C. BILLEY AND BRANT C. JONES 



the height of its row. Each position can be empty, contain a single generator of the heap, or contain the 
two generators sisj in the case when the entry lies in the first column of a type D heap. 
Define two subsets of the heap as follows: 

The lower cone of (a, b): Cone A (a,6) = {(a, b) + -1) + j(2,0) G 1? : i G [0,oo), j G [0,i]}. 
The upper cone of (a, 6): Cone v (a,6) = {(a, b) + 1) + j(2,0) G Z 2 : i G [0,oo),j G [0,i]}. 

After coalescing the heap, the points in a heap that lie in Cone A (a, b) are precisely the elements of 
the heap poset that are smaller than (a, b). Similarly, the points in a heap lying in Cone v (a, b) are the 
elements in the heap poset that are greater than (a, b). 

Definition 7.6. A collection of lattice points L is laterally convex if (a, b) , (c, d) G L implies Cone A (a, b)C\ 
Cone v (c,d) C L. 

Lemma 7.7 (Convexity Lemma). The coalesced heap of a connected convex element in type A, B or 
D is laterally convex. 

Proof. Every minimal pair in a convex element is resolved by two distinct elements x, y. If these ele- 
ments appear on different levels in the coalesced heap then there must be a chain of elements preventing 
the lower element, say x, from rising without raising y. But, this would imply there is an alcove of the 
form 




consisting of a minimal pair without a distinct resolution, contradicting that the heap is convex. There- 
fore, locally every minimal pair is only two rows apart (a, 6), (a, b + 2), and its distinct resolution is on 
adjacent points (a — 1, b + 1), (a + 1, b + 1), given pictorially as 



(7-D 

Suppose the positions (a, b), (c, d) contain entries of the coalesced heap, and (a, b) G Cone v (c, d) and 
(c, d) G Cone A (a, b). Since the heap is connected, there must exist a chain of adjacent non-commuting 
generators connecting the two points in the coalesced heap. Every minimal pair along the path must have 
a distinct resolution which by the argument above looks locally like (7.1). Filling out the diamond for 
each minimal pair recursively implies that every point in Cone A (a, b) n Cone v (c, d) is contained in the 
heap. □ 

Remark 7.8. In the classification of Section 8, we will consider only connected convex elements w, and 
we will assume that they have been embedded in the lattice so that Heap(w) is coalesced. Then, these 
lattice points satisfy lateral convexity by Lemma 7.7. Hence, there is a minimal resolution of any type 
D heap fragment that arises in this fashion, as a subset of lattice points of some Heap(w). In particular, 
we can add lattice points to the heap fragment to resolve minimal pairs of entries over columns 2, . . . , n 
distinctly as in the classical lateral convexity of [BW01], and we may resolve minimal pairs of entries in 
the first column as prescribed by Lemma 6.7. We frequently invoke the Convexity Lemma to speak of 
the resolution by convexity of some collection of entries inside a connected convex non-Deodhar type D 
heap, and deduce the existence of heap entries that were not explicitly given. 

Lemma 7.9. Let w be a connected convex element of type A, B, or D with coalesced Heap(w). Then, 
the point y covers x in the heap poset for w if and only if x has heap coordinates and y has heap 
coordinates (i ± l,j + 1). 
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Proof. Suppose x has heap coordinates and y has heap coordinates (i ± 1, j + 1). Then, x and y 
do not commute, and since the level difference is 1, we have that y covers x in the heap poset. 

On the other hand, if x has heap coordinates but y has heap coordinates (i ± 8\,j + 62) for 
^2 > <5i > 1 with 61 + 62 > 2, then all of the points in C = Cone A (i ± 61, j + 62) fl Cone v (i, j) 
are in Heap{w) by the lateral convexity proved in Lemma 7.7. In particular, z = (i ± l,j + 1) is in 
C C Heap{w), so we have x < z < y in the heap poset, hence y does not cover x in the heap poset. 

If y has heap coordinates (i±6i,j + 62) for <5i > 62, then x and y are unrelated in the heap poset. □ 

Let M^ x y ^ be the number of entries located at position (x, y) in the coalesced heap for w. These 
occupation numbers can take the values 1 or 2 in type D, but the value 2 can only appear in column 1 
and must obey the rules in Lemma 6.7. 

Definition 7.10. Let w and p be connected convex elements of type A, B or D with coalesced heaps 
Heap(w) and Heap(p), respectively. We say that Heap(w) contains Heap(p) as a saturated subset 
of lattice points, if there exist offsets i,j G Z and a Coxeter embedding / such that for all occupied 
positions (x, y) in Heap(f(p)), we have 

M (z,y)+i(2,0)+j(l,l) = M (x,y) 

Example 7.11. Letp = S2S1S3S2 and w = S2S\s^s^S2- Note that 

is not saturated in •• • 

because M? 1 1} = 1 while 1} = 2. 

On the other hand, Heap(w) is saturated in the heap of any w' containing s2S1Sj.s3.s2 as a factor. 

Lemma 7.12. Let w and p be connected convex elements of type A, B, or D with coalesced heaps 
Heap(w) and Heap{p), respectively. Then, the following are equivalent: 

(1) w contains p as an embedded factor. 

(2) Heap{p) is contained in Heap{w) as a saturated subset of lattice points. 

Proof. (1) =^ (2). Recall from [Sta97] that a subposet Q of some poset R is called convex if b G Q 
whenever a < b < c in R and a,c G Q. Take a reduced expression for w of the form w = u/(p)v where 
/ is a Coxeter embedding for p. By Definition 3.1, the labeled heap poset of w = u/(p)v contains the 
labeled heap poset of /(p) as a convex subposet. Lemma 7.9 shows that the Hasse diagram for the heap 
poset of w can be embedded into [n] x N where (i, j) < (i ± 1, j + 1) is the corresponding cover relation, 
and this embedding is Heap(w). Hence, the convex subposet corresponding to the labeled heap poset of 
/(p) appears in Heap{w) as a saturated subset of lattice points with shape Heap(f(p)). 

(2) (1). If Heap(f(p)) is contained as a saturated subset of the lattice points in Heap(w) 

for some Coxeter embedding /, then we can build Heap(w) from Heap(f(p)) by sequentially adding 
lattice points. Moreover we can only add points that are maximal or minimal entries of the intermediate 
heap since Heap(f(p)) is saturated in Heap(w). This operation is equivalent to multiplying f(p) on the 
left or right by the respective generators. Hence, w contains f(p) as a factor. □ 

We now generalize the notion from [BW01] of a right critical zero associated to a defect for convex 
heaps in type D. 

Definition 7.13. The first zero encountered by the right string of a defect along the southeast diagonal 
containing the defect is the right critical zero. 
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For example, the defect d has right critical zero r: 




Lemma 7.14 (Right Critical Zero Lemma). Suppose w € D n is convex, and a is a mask on w. Then 
every defect in Heap{w) has a right critical zero. 

Proof. Suppose the right string for a defect never encounters an entry with mask-value as it travels 
southeast. It eventually turns southwest when it hits the first entry (a, b) not in the heap along that 
diagonal. As the string travels down, it cannot encounter another heap element in column a — 1 before 
encountering one in column a by convexity. If an element is encountered in position (a, b — i) as the 
string travels down, then by lateral convexity (a, b) must exist in the heap contradicting our hypothesis. 
If no element is encountered in position (a, b — i) as the string travels down, then it cannot cross the left 
string below, contradicting the fact that the original position was a defect by Corollary 6.3. □ 

Note that defects in type D need not have a corresponding left critical zero, since a string leaving the 
heap on the left can re-enter the heap without a mask-value entry. See Example 6.2. 



8. Classification of convex type D patterns 

In this section, we show that among the set of convex elements of type D, there are precisely six 
that are minimally non-Deodhar under the partial order of embedded factor containment. Recall that 
the convex elements are short braid avoiding, hence they have unique heaps. These minimal heaps 
have the property that if we remove either an entry that is minimal or maximal with respect to the heap 
poset structure, then the resulting heap corresponds to a Deodhar element. Furthermore, we will always 
consider the heaps to be coalesced as described in Definition 7.5. 

Theorem 8.1. Below is the complete list of the heaps of convex minimal non-Deodhar embedded factor 
patterns in type D up to the Coxeter graph isomorphism that interchanges s± and sj: 



(8.1) 
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FLHEX 


HEX 


HEX 2 


HEX^a 


HEX 3b 


HEX A 



Each heap above is decorated with a mask that demonstrates the non-Deodhar condition using the 
notation described in Section 6. It is straightforward to verify by computer that these heaps are minimal 
among the convex non-Deodhar elements in type D using embedded factor containment. Our goal is to 
show that this list is complete. Note that we could reduce the list further by requiring both w and w" 1 to 
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avoid the pattern HEX^ a since HEX^ is its inverse and by Lemma 5.1, w is non-Deodhar if and only 
if w~ 1 is non-Deodhar. All the other heaps in (8.1) have w = w~ 1 . 

Note that the pattern HEX can embed on any 7 connected columns corresponding to a path of length 
7 in the Coxeter graph, including sj, S2, ■ ■ ■ , sj. The other patterns are tied to the initial segment of the 
type D Coxeter graph, since they include both of the generators s\, sj from column 1. These patterns 
are the heaps for the reduced words that appear in the table on Page 2 in the type D case. 

Remark 8.2. Recall from Lemma 7.12 that factor containment is expressed in the heap setting by deleting 
a sequence of entries that are minimal or maximal entries of the intermediate heap at each step. In 
particular, HEX is not a saturated subset of FLHEXq by Definition 7. 10. 

Let D„ and Ag denote the convex elements in type D n and type A on any finite number of generators, 
respectively. We will prove Theorem 8.1 by constructing a pair of projection maps 

■kne -D n -* Ag 
kse -D n -* A^ 

that we can use to compare minimal patterns in type D with the hexagon in type A. We will show that 
these maps have have the following properties: 

a. If w G D~l is non-Deodhar in type D, then at least one of the two projections sends w to a 
non-Deodhar element in type A. 

b. If ttne{w) or ttse(w) is non-Deodhar in type A, then w contains one of the patterns listed above 
as an embedded factor. 

Therefore, if w is any minimal non-Deodhar element in type D, then it appears on the list. Hence, the list 
of convex minimal non-Deodhar elements is complete. In particular, all convex minimal non-Deodhar 
patterns occur in Dg. 

We will define the projections itne{w), ttse{w) in terms of three other maps. First, define the map 

<Pi : D% — ► A n -i for i € {1,1} 

as the projection of w onto a type A heap obtained by removing all the entries that correspond to the i 
generator, where i is the generator {1, 1} \ {i}. That is, ipi assigns the Sj generator from column 1 of 
the type D heap to si in column 1 of the type A heap, and removes all the entries from column 1 that 
correspond to the other generator. This generally leaves a heap fragment in type A that is not convex. 

How might we construct an associated convex element in type A? We consider two constructions. We 
could take an additive approach, denoted 

add; : D% Ag, 

where we first project the element into type A by the map ipi, then extend the Coxeter graph linearly 
to the left beyond the first column and add entries as necessary in order to resolve all minimal pairs of 
entries that do not have a distinct resolution. By Remark 7.8 restricted to type A, this operation is unique. 
For example, the composition shown below is addi. 



• • • • • • 

— j. * • • 

• • • 

• • • • • • 

Here, * denotes a point not in the heap, and * denotes a highlighted point in the heap. 
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The second construction is a subtractive approach that applies when there are at least two entries in 
the first column of the type D heap. Define 

suh NE :D%^D% 

where subjv^j removes entries in the northeast (NE) diagonal up to the first lattice point not in the heap. 
Similarly, subg^ removes the southeast (SE) diagonal. This process is referred to as "shaving off a 
diagonal" in the proofs that follow. For example, 



sub 



SE 



* • 

* 



Note that the result of applying sub^s or subge is always again convex because we are removing 
entries that are maximal or minimal with respect to the heap poset, so the entries cannot be used in a 
distinct resolution. 

We now define ~kse{w) and ttne(w) in cases depending on how many levels the coalesced Heap(w) 
contains in the first column. The rough idea that motivates the following technical definition is to use the 
minimal patterns listed in (8.1) as a guide, taking a subtractive approach when we have a heap fragment 
that is Deodhar and taking an additive approach when we have a heap fragment that is non-Deodhar. 

Definition 8.3. Let w 6 D% . Suppose that the first column of Heap(w) has entries in k(w) distinct 
levels. Let i £ {1, 1} be the generator corresponding to the top entry in column 1. Let dir denote the 
direction SE or NE. Then, we define ir^r ■ — > as follows: 



W 

addj(w) 
addi(u?) 

sub dir {w) 

addi(io) 

snh dir {w) 

addj(w) 

ir dir o suh dir (w) 

addifu?) 



if k(w) = 0. 

if k(w) = 1 and there is a single entry in column 1. 
if k(w) = 1 and there are two entries in column 1, 

with at least 4 entries in column 4. 
if k(w) = 1 and there are two entries in column 1, 

with at most 3 entries in column 4. 
if k(w) = 2 and there are at least 4 entries in column 3. 
if k(w) = 2 and there are at most 3 entries in column 3. 
if k(w) = 3. 
if k(w) = 4. 
if k(w) > 5. 



This definition relies on the following observations. 

(1) When there are entries in more than one level of column 1, the entries of the first column must 
alternate by Lemma 6.7. 

(2) In the case k{w) = 1, if there two entries in column 1 with less than 4 entries in column 4, then 
column 1 contains both s\ and sj on the same level in column 1. We define the map -K dir to 
remove both of these generators in column 1, and shave any remaining entries from the chosen 
diagonal. 
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(3) In the case k(w) = 4, observe that TVdir is defined by shaving the top or bottom entry of column 
1, and then reducing to the case when k(w) = 3. 

Note that the map TTdir always preserves the occupied lattice points that are present in the type D heap, 
with the possible exception that they may remove the northeast diagonal of maximal entries or southeast 
diagonal of minimal entries, starting from the first column and working to the right. 

The -Kdir projections satisfy a useful property with respect to taking inverses that we will exploit in the 
classification. 

Lemma 8.4. For all w E D%, we have 

Proof. Observe that the heap of w^ 1 is obtained from the heap of w by flipping the diagram upside down. 
In particular, the number of entries in each column of w and w^ 1 is the same. Therefore, w^ 1 falls into 
the same case as w when computing irdir- Flipping the heap has the effect of switching all the SE and 
NE diagonals, while the additive resolutions remain the same. □ 

Consider the following partial heaps drawn with black dots called the I-shape and the 4-stack: 

• • • • 




I-shape 4-stack 

Some of the shaded dots will also appear in any convex heap containing either the I-shape or the 4-stack. 
However, if the shape appears in the first 4 columns then some of the shaded dots on the left hand side of 
the picture will get trucated in the heap. Note that the 4-stack consists of 4 adjacent copies of the same 
generator if it appears in column 1. Therefore, a "4-stack" in column 1 of a convex heap necessarily 
contains entries from 7 distinct levels. The following result is used frequently in the classification. 

Lemma 8.5. (Shape Lemma) Suppose w £ D^, Heap{w) contains an I-shape or a 4-stack, and the 
first column of Heap{w) has entries from at least two distinct levels. Then, w is non-Deodhar and there 
is a choice of projection -kne{w) or t:se{w) that contains a hexagon, and so preserves the non-Deodhar 
condition. 

Proof. Consider the convex resolution of either the I-shape or the 4-stack. By Remark 7.8, the points in 
the convex resolution must appear in Heap(w). This implies that all of the gray dots shown in (8.2) to 
the right of the shape and a subset of the gray dots to the left of the shape must be in the heap, depending 
on how far the shape lies from the first column. By considering the columns where the shape lies, one 
can verify that no matter where the I-shape or 4-stack appeal - , the convex resolution must contain one of 
the patterns in (8.1) as a set of points in the lattice. Since Heap(w) contains entries from at least two 
distinct levels, we have that Heap(w) contains one of the minimal heaps in (8.1) as a saturated set of 
lattice points by Lemma 6.7, hence w also contains it as an embedded factor by Lemma 7.12. Therefore, 
w is non-Deodhar. 

Furthermore, it can be verified that Definition 8.3 has been chosen precisely so that if Heap(w) 
contains one of the heaps in {HEX, HEX2, HEX^ a , HEX^, HEX^} as an embedded factor, then 
there is always a choice of direction NE or SE so that the projection ■n^ir contains a hexagon in type A. 

To carry out the verification, suppose that Heap(w) contains HEX using column 1. Then, if there 
are two entries in column 1, we either project additively (if there is also a 4-stack in column 3), or we 
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choose to shave the entry of column 1 that is not being used in the hexagon, and its diagonal. If there 
are 3 entries in column 1 , then we always project additively, so the hexagon is preserved. If there are 4 
entries in column 1, then we shave one of the extremal entries that is not used in the hexagon, and project 
additively. If there are more than 4 entries in column 1, then we project additively so the hexagon is 
preserved. If the first column of the HEX factor occurs in column 2 of Heap(w), or further to the right, 
and we are taking a subtractive resolution in tt^^w), then there is always a diagonal to shave that does 
not intersect the hexagon. Otherwise, we take an additive resolution, so the hexagon is preserved. 

The shapes {HEX2, HEX^ a , HEX^, HEX4} must all occur in column 1, so it suffices to check 
that there is a choice of projection that preserves the shape, regardless of how many entries column 1 
contains. Note that column 1 must contain at least two entries by hypothesis, and if it contains more than 
4 entries, then we resolve additively, so the shapes are preserved. The cases left to verify are then that 
there exists a non-Deodhar projection ttne{w) or ttse{w) if Heap{w) contains: 

• HEX2 with 2, 3 or 4 entries in column 1, 

• HEX^a with 3 or 4 entries in column 1, 

• HEX^ with 3 or 4 entries in column 1 , or 

• HEX4 with 3 or 4 entries in column 1 . 

This verification is finite and straightforward, so we omit it. □ 

We now prove Property (b) for 7r^ ir . 

Proposition 8.6. Let w £ D^. If ~Kdir{w) € is non-Deodhar, then w contains a pattern from (8.1) 
as an embedded factor. 

Proof. Suppose itdiriw) is non-Deodhar. Then, by Theorem 5.9 the heap of iTdir{w) contains a hexagon. 
We begin by considering the various ways in which the first column of the type D heap can be shifted 
under the -Kdir map. Let a(w) be the number of the column that contains the image of column 1 under 
ftdir(w). This can be computed by determining how many columns must be added to the left when 
applying the add map or subtracted if the entire first column is removed. 



Case for -Kdir{w) 


Shift a(w) 


k(w) = 


1 


k(w) = 1 and there is a single entry in column 1 


1 


k(w) = 1 and there are two entries in column 1, with at least 4 


1 


entries in column 4 




k(w) = 1 and there are two entries in column 1, with less than 4 





entries in column 4 




k(w) = 2 and there are at least 4 entries in column 3 


2 


k(w) = 2 and there are less than 4 entries in column 3 


1 


k(w) = 3 


3 


k(w) = 4 


3 


k(w) > 5 


k(w) 



If the hexagon appears weakly to the right of column a(w) then Heap(w) contains the shape HEX 
in (8.1), since -Kdir only adds points to the left of column a{w). In particular, if a(w) = 0, then from the 
table above we see k(w) = 1 and column 1 contains {s\, sj} on the same level, with less than 4 entries 
in column 4. In this case, ~Kdir{w) removes all of the entries in the first column, and shaves a diagonal. 
If there exists a hexagon in -Kdir{w), we have that there is a hexagon in Heap(w) that lies strictly to the 
right of column 1 in Heap(w), so w contains HEX as an embedded factor. 

If a(w) = 1 then from the table above we see that k{w) = 1 or k(w) = 2 and there are less than 
4 entries in column 3. In each of these cases the 'Kdir projection doesn't add any points to the heap. 
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Therefore, if 71"^ (u>) contains a hexagon that uses column 1, then Heap(w) contains either HEX or 
FLHEXq as an embedded factor. 

If a(w) = 2 then k(w) = 2 and there is a 4 stack in column 3, so by Lemma 6.7 and Remark 7.8, 
Heap(w) contains HEX2 as an embedded factor. 

If a(w) = 3, then k{w) = 3 or k(w) = 4. In the case k(w) = 3, irdir{w) adds three new points to the 
left of Heap(w). If a hexagon in 7Tdir(w) uses column 1, then Heap(w) must contain HEX4. 



Kdir(w) • # • * 



• • • • • 

• • • • 



If a hexagon in ~Kdi r {w) uses column 2 (but not column 1) then Heap(w) contains HEX^ a or HEX^ 
as an embedded factor. If a hexagon appears in column 3 or further to the right, then Heap(w) contains 
HEX as an embedded factor as well. 

In the case <j(w) = 3 and k(w) = 4 then 7r^ r (to) shaves a maximal or minimal diagonal and adds 
back three points as above. The same analysis of the placement of a hexagon in 7Tdi r {w) implies the 
existence of a HEX^ a , HEX^ or a HEX4 as embedded factors. 

If a(w) = s > 5, we again must have at least 5 entries in column 1 of w, so by convexity w contains 
HEX4. Note that a(w) = 4 never occurs. □ 



We now turn to the proof of Property (a) for iTdir- In preparation for the proof, recall from Corollary 6.3 
that the strings for a defect must cross below the defect. The first time the two strings meet again will be 
called the initial string crossing for the defect or just the string crossing, for short. 

Proposition 8.7. Let w 6 D^. If w is non-Deodhar, then there is some choice of projection ttne{w) or 
1{Se{w) that is non-Deodhar. 

Proof. To prove this, we will consider the various cases given in Definition 8.3 for 7r^ r , breaking on 
the number of levels in the first column of w denoted k(w). Recall from (2.5) that a proper mask is 
non-Deodhar whenever 

# zero-defects > # plain-zeros 

which we refer to as the non-Deodhar bound throughout the proof. The general strategy is to either 
show that w contains a 4-stack or an I-shape from Lemma 8.5 when k(w) > 2, or to assume a proper 
non-Deodhar mask on w, and then show how to adjust it to obtain a mask on ~Kdi r (w) that retains the 
non-Deodhar bound by deleting at least as many plain-zeros as zero-defects. 



Case k(w) = or 1. First, suppose that w has at most a single entry in the first column so 7r^ r is just 
the embedding map into type A. Then we can interpret w as an element of type A by projecting the 
unique generator in the first column to the generator s\ in type A. Hence, w is non-Deodhar if and only 
if it contains a hexagon if and only if ~Kdir(w) contains a hexagon, so 7r^ r preserves the non-Deodhar 
condition in this case. 

Second, suppose that w has two entries lying at the same level in the first column and there are 4 
entries in column 4. Then kne = kse = fi just removes the single sj entry, and by lateral convexity 
TTdir{w) contains a hexagon. 
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Third, if there exist two entries in column 1 and there are fewer than 4 entries in column 4, then at 
least one of the points labeled x or x' is not in the heap: 




By Lemma 5.1, we have that w is non-Deodhar if and only if w is non-Deodhar. Hence, by Lemma 8.4 
we may assume that the entry x' is not in the heap, by taking a non-Deodhar mask on w~ 1 if necessary, 
whose heap is obtained from Heap(w) by flipping the diagram upside down. In this case, we choose 
the projection -kne{w) so Heap(TTNE(w)) is obtained from Heap(w) by shaving off the maximal NE 
diagonal up to x'. 

We need to verify that the non-Deodhar bound holds on the mask restricted to ttne(w)- By convexity, 
there are at most three occupied positions in the NE diagonal as in the figure above since x' is not in 
the heap. There cannot be a defect for w in column 1 since the defect would have no critical generator 
so there can be at most two defects along the maximal NE diagonal. If there exists a defect whose left 
string touches either entry in the first column, then both entries in the first column must have mask value 
0, for otherwise the left string of the defect would exit the heap without crossing the right string, or be 
labeled negatively. There could be a single defect on the NE diagonal in column 3 whose left string does 
not touch the entries in column 1 , but then it must intersect a plain zero along the NE diagonal in column 
2. Therefore, shaving off the NE diagonal removes at least as many plain-zeros as zero-defects, so the 
Deodhar bound is preserved. 



Case k(w) = 2. By Lemma 6.7, whenever the first column contains entries on more than one distinct 
level, we must have that each level contains a unique entry, and the entries alternate between the s\ and 
sj generators. Without loss of generality, we can assume assume the top entry in column 1 is sj. 

First, suppose k(w) = 2 and there is a 4-stack in column 3. By the Shape Lemma 8.5, there is a 
hexagon in Heap(7r,n r {w)). 

Second, suppose that there are at most 3 entries in column 3. Then we can further assume that the 
maximal NE diagonal from the top entry in column 1 contains at most two entries by taking a non- 
Deodhar mask on w~ l if necessary using Lemma 5.1 and Lemma 8.4. The projection ttne(w) shaves 
off the NE diagonal, so we need to show that this removes at least as many plain-zeros as zero-defects 
to preserve the bound. There cannot be a defect in column 1 , because the defect would have no critical 
generator, so there can be at most one defect in the maximal NE diagonal, located in position a: 



* 




* 

-k • 
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If x has mask value 1, then the left string of a is negatively signed while the right string has a positive 
label, in which case a is not a defect by Lemma 6.1. Hence, if a is a zero-defect, then x must be a 
plain-zero so removing both x and a from the heap will preserve the non-Deodhar bound. 

Case k(w) = 3. Suppose that w has entries on 3 distinct levels in the first column and assume without 
loss of generality that the top entry corresponds to s\. The projection ttne = ^se adds the three entries 
marked as ★ to the left of Heap(w): 



•k • 
* • 



If w contains a 4-stack in column 2, then ttne{w) contains a hexagon by the Shape Lemma 8.5. 
Hence, we can assume there are at most 3 entries in column 2. By considering w' 1 if necessary, and 
applying Lemma 5.1 and Lemma 8.4, we can assume that the point marked c in column 2 is not in the 
heap. 

Fix a non-Deodhar mask for w. Since there are exactly three alternating entries in the first column, 
there is at most a single defect in the first column, and it must occur in position d because any defect 
requires a critical generator below it. If d is not a defect then extend the mask to ttne{w) by setting the 
mask- values of the three new points to 1 : 



(8.3) * c , , * c 

★ * 



If a string passes through point z in Heap(w), it changes sign and so it cannot be the left string of 
any defect. Therefore the mask assignment in (8.3) preserves all zero-defects and plain-zeros, hence it 
preserves the non-Deodhar bound. 

Now, suppose that d is a defect in the first column. Then we have the following heap fragment where 
the critical generator below d must have mask- value 1, but the mask- values of entries x, y and z are 
variable: 



Breaking into cases on the mask-values of y and z, we demonstrate how to extend the given non- 
Deodhar mask of w to a non-Deodhar mask of ttne(w). 
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If z has mask- value 1, then y must have mask- value or else the right string for d will be labeled 
positively, while the left string will be labeled negatively, contradicting that d is a defect. In this case, the 
strings for d cross at z, and both become negatively labeled, so must cross once more below z. Thus, we 
can set the mask-values of the additional entries to the left as shown: 




The string dynamics show that d and e are defects in Heap{iTNE{w))- The new mask has one additional 
plain-zero and zero-defect so the non-Deodhar bound is maintained. 

If both y and z have mask- value 0, then the strings for d must cross below x in the heap poset. In this 
case, we can set the mask-values of the new entries as shown: 




The string dynamics for d are preserved in the new mask, so d remains a defect, and z becomes a defect 
since it has exactly the same string dynamics as d. Note that z is not a defect for w since it corresponds 
to the sj generator in Heap(w). This mask is non-Deodhar because the original one was; we've changed 
a plain zero to a defect and added two plain zero entries, maintaining the non-Deodhar bound. 

If z has mask- value and y has mask- value 1, then either there is another defect e whose left string 
touches z, or there is not. If not, we can just move the zero at z to the left to preserve the string dynamics 
for d and extend the mask to ttne(w) as shown: 
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This mask is non-Deodhar assuming the original one was since we have not changed the number of 
zero-defects or plain-zeros. 

On the other hand, if z is touched by the left string of a defect at or above e, then we claim Heap{w) 
also contains an entry p below x in column 3. Hence, the heap contains an I-shape with corners c, d, e 
and p, so the Shape Lemma 8.5 implies that ttne(w) is non-Deodhar. For example, 




To verify the existence of p E Heap(w), observe that the highest possible crossing for the strings of d 
is at the point c, and consider two cases. If the strings of d do not cross at c, they must cross at a point 
below c and p so by convexity p € Heap(w). For the strings of d to cross at c, the point x must have 
mask-value 1 and the right critical zero of d is located as shown, in which case the strings for the other 
defect e cross at or below the point p, so again p € Heap(w). 

Thus, for all possible mask- values of x,y the projection itne preserves the non-Deodhar condition, 
finishing the case k(w) = 3. 

Case k(w) = 4. Suppose that w has entries on 4 distinct levels in the first column. 
First, we reduce to the case where w contains a decorated heap fragment of the form: 

(8.4) 

o a *y' 
p *«' 




Then we study cases corresponding to the mask values in the gray star positions. 

If there is a point at x or x' , then there is a 4-stack in the second column. If there is a point at y or y', 
we obtain an I-shape with the other entries that exist by convexity. In either case, the Shape Lemma 8.5 
implies vr^ r (w;) contains a hexagon. Similarly, if both v and v' exist, then we obtain an I-shape, so at 
least one of them is not in the heap. By considering w^ 1 if necessary, we can assume that v' is not in the 
heap. 

By Definition 8.3 for iTdir> we must first choose one of the extremal diagonals to shave, and then 
additively resolve the remaining heap fragment, in such a way that the non-Deodhar bound is maintained. 
Hence, choosing a projection ttme or tt$e really just amounts to choosing the top or bottom entry from 
the first column to remove in such a way that the non-Deodhar bound is maintained. Once we choose the 
entry and shave it, we will have a non-Deodhar heap with only three entries in the first column, so we 
can appeal to the previous case k(w) = 3 when we apply n^ ir the second time. 
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If z has mask-value 0, then it can play no critical role in any defect: it cannot be a defect itself since 
there is no critical generator below, and it cannot be a critical-zero for a defect since both strings meeting 
at z leave the heap below. Therefore, we choose to shave z in this case which maintains the non-Deodhar 
bound. 

Hence, we can assume that z has mask-value 1 and that it is a string crossing for some zero-defect, 
since we can again remove it if it is not. Note that if a is not a zero-defect, we can choose to shave 
it without changing the Deodhar bound. Therefore, we can also assume that a is a zero-defect, which 
forces c to have mask-value 1 since it must be the critical generator for a. Thus, we have a heap fragment 
of the form in (8.4). 

Suppose that b is a plain zero. Then, the strings that cross at z must both be labeled negatively every- 
where above z since there is no other active sj generator to change the signs of the strings. Therefore, no 
defect above z can be removed by removing z since the strings remain in increasing order as they pass 
through z and all defects in this case must correspond with one of the type A generators, s\,S2, ■ ■ ■ , s n -\. 
Hence, we can shave z without affecting the defect status of any defect whose strings cross at z, and so 
we maintain the Deodhar bound in this case. 

Next, suppose b is a zero-defect. Then its strings must cross at z since this is its critical generator, 
and there are no lower entries. Moreover, the right critical zero for a must occur at p, or the strings for a 
will never cross. Hence, the paths of the strings from b are completely prescribed and we have the heap 
fragment: 



o 



o 



Since p cannot be a defect (because it lacks a critical generator), and it cannot enable a left string crossing 
for any defect above p (or we introduce an I-shape), removing a and changing the mask-value at p to 1 
maintains the Deodhar bound. 

Finally, suppose that b has mask- value 1 , and consider the possible locations of the zero-defect whose 
strings cross at z; call it d z . If d z is located on the NE diagonal from c, then its left string is labeled 
2, which is a contradiction. If d z is located at p, then the remaining mask values are forced, and in 
particular, q must have mask-value 1 as shown in Figure 2(a). Then, we obtain the contradiction that a 
cannot be a defect in this case since its left string is labeled 2, while its right string is labeled 1. Thus 
d z = a is the only viable possibility. 

Suppose d z = a. If the right critical zero for a occurs in column 4, then the mask- values are forced 
and in particular q must be a plain-zero as shown in Figure 2(b). Hence, we can shave a and change 
the mask- value of q to 1, since q is not itself a defect, nor can any other defect use q to effect a string 
crossing. This choice of projection preserves the Deodhar bound. 

If the right critical zero for a is located in column 2 or 3 and it is a plain-zero, then we can shave a 
and change the mask- value of the right critical zero to 1, since no other defect can use the right critical 
zero of a to effect a string crossing. Hence, we may assume that the right critical zero of a is itself a 
zero-defect. 

If the right critical zero for a is in column 3, then we have one of the cases shown in Figure 3. Once 
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(a) defect at p (b) defect at a with right critical zero in column 4 

Figure 2. Locations of defects with strings crossing at z 




(a) q has mask-value 1 (b) q has mask-value 

FIGURE 3. a has a right-critical zero that is a defect in column 3 

we choose a mask- value for q, the rest of the mask- values are determined. In each case, the right critical 
zero for a cannot be a defect because its strings do not cross, which is a contradiction. 

If the right critical zero for a is in column 2, then we have the cases shown in Figure 4. If q has 
mask-value 1 then the strings for a cannot cross at z, and if q has mask-value 0, then the right critical 
zero for a cannot be a defect because its strings do not cross. 

Thus, in all cases there is a choice of projection ttne or ttse which preserves the non-Deodhar con- 
dition. 

Case k(w) = 5. Suppose that w has entries on 5 or more distinct levels in the first column. Then, w 
contains the pattern HEX4 and so the projections 7Tdir(w) contain a hexagon. 



We have shown in all cases that there is a choice of projection ttne(w) or ttse(w) that remains 
non-Deodhar if w is non-Deodhar, concluding the proof. □ 
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(a) q has mask-value 1 (b) q has mask-value 

Figure 4. a has a right-critical zero that is a defect in column 2 

9. Proof of the type D characterization theorem 
In this section, we complete the classification of the minimal non-Deodhar embedded factors for type 

D. 

Theorem 9.1. Suppose w is a short braid avoiding, type D element that is not convex. Then, w is 
non-Deodhar if and only ifw contains [16785234] as a 1-line pattern. 

Proof. Since w is not convex, we have that Heap(w) contains a minimal pair of entries that require a 
left resolution. 

Suppose w is non-Deodhar. By Lemma 6.8, Heap(w) must have an sisj entry. By Lemma 6.10, w 
contains a convex non-Deodhar element w as a 1-line pattern. Moreover, the construction of w preserves 
the sisj entry. The only convex minimally non-Deodhar heap from Theorem 8.1 that has an entry with 
an s±si entry is: 

o 

o o 
o o o 



FLHEXq = tfeap( [16785234]) 



•• • 

o 



Hence, w must contain a FLHEXq factor. Note the relative order of the 8 strings passing through a 
FLHEXq gives rise to a [T6785234] pattern in w. The relative order of these strings cannot change 
due to multiplication by additional generators above or below in a short braid avoiding heap since any 
additional adjacent transposition applied on the left or right of [T6785234] would create a short braid. 
Therefore, both w and w contain [T6785234] as a 1-line pattern. 

Conversely, suppose that w is a short braid avoiding type D element that contains [16785234] as a 1- 
line pattern. We will reduce to the case where w± = 1 and the values in the pattern representing 6785234 
occur consecutively in the 1-line notation for w. Then, we will be able to give a reduced factorization for 
w that contains a FLHEXq which is known to be non-Deodhar; see Example 5.11. 

Consider the 1-line notation for w and highlight a pattern instance for [T6785234] on values a\ < 
a 2 < . . ■ < as and in positions p\ < . . . < ps : 



w = [. . . a\ . . . Ct6 • • • «7 • • • as ■ ■ ■ o>5 ■ ■ ■ a>2 ■ ■ ■ a>3 ■ ■ ■ o>4 ■ ■ ■ ] 
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Since all 5 of the signed permutations 



[T23],[213],[T32],[23T],[312] 



with at least one ascent and all entries negative are forbidden by Theorem 7.3, we have that all of the 
entries besides 01,05 must be positive. In addition, since [123], [213] and [231] are all forbidden by 
Theorem 7.3, we have that ai = 1. Hence, if we let i = pi then w contains w' = f0Sj_iSj_2 • • • si as 
a factor, and w' also contains the 1-line pattern [T6785234] with w[ = 1. Therefore, we only need to 
consider the case when w\ = a± = 1. 

Next, if there exist entries in w between 05 and 02, let t = p$ — 1 be the position just to the left of 02- 
By Theorem 7.3, o§ > w t > 02 would imply a forbidden [321] pattern, so either 

(1) wt > as in which case w' = wst also has the pattern and is a factor of w, or 

(2) 1 < wt < «2 in which case we can change the highlighted pattern to choose a pattern instance 
where the 02 is closer to the 05. 

Therefore, we only need to consider the case when 0502 are consecutive. Furthermore, by the same 
argument in which the role of 02 is replaced by 03 and 04, respectively, we can assume 05020304 are 
consecutive in w. Note also that all of the entries in positions > ps must have values > 04, for otherwise 
we obtain a forbidden [321] instance, with a§ > 04. 

By Theorem 7.3, w avoids the patterns [213], [312], [321] so the values in w between the 1 and 05 are 
increasing positive numbers. By changing the highlighted pattern instance if necessary, we can assume 
the values 060708 are the biggest three among these so 06070305 are in consecutive positions. Consider 
the value in position P2 — 1, just to the left of a§. If 05 > w P2 -\ > 02 then w contains the 1-line pattern 
[231] which is forbidden by Theorem 7.3. If w P2 ~i > 05 then we move the as out of the pattern to the 
right, and change the highlighted pattern so that w P2 ~i becomes 06, 06 becomes 07, and 07 becomes as- 
Specifically, if i = p^ then w contains w' = wsiSi + iSi + 2Si + 3 as a factor and w' also contains [16785234] 
as a 1-line pattern in the manner described. Hence, we only need to consider the case when the entries 
in w between 1 and a§ all have values less than 02- 

Summarizing, we can assume the 1-line notation for w is of the form [T . . . 06^70805020304 . . . ] with 
the following conditions: 

(1) The elements in the first dotted sequence are increasing, all with positive values less than 02- 

(2) All entries in the second dotted sequence have value greater than 04 > 0. 

When we draw the string diagram corresponding to these rules, we find that w contains FLHEX^ as 
a factor. 

Alternatively, we obtain a reduced factorization of w containing a non-Deodhar factor as follows. Let 
1 < yi < ■ ■ ■ < y 3 < 05 be the values that appear to the right of 05 in w. Note that 02, 03, 04 are the 
smallest three of the y^'s. Let u be the permutation with values 06, 07, as moved left and consecutively 
stacked adjacent to 05 and with values y\,...,y^ moved right and consecutively stacked adjacent to 05 , 
i.e. 



for some permutation v that arranges the entries to the right of 04 in their final order. Here k = 05 — 2 — j 
is the number of strings strictly between 1 and 05 that remain between positions 1 and p^. The FLHEX^ 
pattern represents the operation that starting from u' moves 05 to position 2, shifts the block 020304 to 
the right of 050703, applies sisj to change the signs on 1 and 05 in positions 1 and 2, and finally moves 




n]. 



n]. 



05 back to position k + 3. 



□ 
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Theorem 9.2. In type D, we have that w is Deodhar if and only if it avoids the 1-line pattern [16785234], 
and the six embedded factors 

S\S 2 S\ (short braid) 
S5S6S7S3S4S5S6S2S3S4S5S1S2S3 (HEX) 
S3S4S5S6S2S3S4S5S1S2S3S4S1S2S3 (HEX 2 ) 
•S4S5S6S2S3S4S5S1S2S3S4S1S2S1 (HEX 3a ) 
S1S4S5S6S2S3S4S5S1S2S3S4S1S2 (HEX 3b ) 
S3s4s5s1s2.s3s4sjs2s3.s1 (HEX4). 



Proof. If w contains a short braid then w is not Deodhar by Corollary 5.3. If w is convex, then w is 
Deodhar if and only if it avoids the patterns HEX, HEX 2 , HEX 3a , HEX 3b , HEX 4 by Theorem 8.1. 
If w is short braid avoiding and not convex, then w is Deodhar if and only if it avoids [16785234] by 
Theorem 9.1. □ 



10. Deodhar elements of exceptional Weyl groups 

The minimal non-Deodhar patterns for the other finite Weyl groups are computed by software that 
implements a game of K. Eriksson described in [Eri95] and [BB05]. We give the minimal lists for each 
type that account for Coxeter graph isomorphisms and patterns contained in parabolic subgroups. For 
example, a Deodhar element of E% must avoid the short braid on each pair of non-commuting generators, 
the hexagon pattern of length 14 contained in the type A parabolic subgroup, the type D patterns, and all 
of the patterns in Eq and E 7 , the former of which can be embedded in two different ways. 

Theorem 10.1. Below is the complete list of minimal non-Deodhar embedded factor patterns in the Weyl 
groups of type E. The only minimal non-Deodhar elements in types G 2 and F4 are short braids. 



Lie type 


Coxeter graph 


Reduced expression patterns 


E 6 


•5 

•0 »1 »2 *3 *4 


S0S1S2S5S3S4S2S3S1S2S5S0S1 

S5SlS2S 3 SoSiS 2 S 5 S4S 3 S 2 SiS 
SlS 2 S 5 S 3 S4S 2 S 3 SlS 2 S 5 S SlS 2 
S2S5S1S2S3S0S1S2S5S4S3S2S1 


E 7 


•5 

•0 *l *2 *3 *4 *6 


S0SlS 2 S 3 S4S 6 S 5 S 2 S 3 S4SlS 2 S 3 S Sl 
S 3 S4SQSlS 2 S 3 S SlS 2 S 5 S 4l S 3 S 2 SlS 
SiS2S3S4S 6 S5S2S3S4SiS2S 3 SoSiS2 
S2S 3 S4S 6 SlS 2 S 3 S SlS 2 S 5 S 4l S 3 S 2 Si 
S5S2S 3 S4SqSiS 2 S 5 S 3 S4S 2 S 3 S SiS 2 S 5 



Proof. These groups are finite, so this list of minimal patterns is verifiable by computer. The code used 
is available at http : //www .math. Washington. edu/~brant/liberikson. html. □ 



EMBEDDED FACTOR PATTERNS FOR DEODHAR ELEMENTS IN KAZHDAN-LUSZTIG THEORY 



37 



11. Patterns of convex elements 

In this section, we prove that the Deodhar property can be characterized by avoiding finitely many 
1-line patterns in types A, B and D. In general, properties characterized by 1-line pattern avoidance are 
not equivalent to properties characterized by embedded factor avoidance, even if the embedded factors 
are convex, as seen in the example below. However, we describe a finite test for when we may translate 
between the two types of pattern avoidance on convex elements. Although it is not a direct generalization, 
this idea is related to a type A result of Tenner [Ten06b]. In [Jon07] the main result of this section has 
been extended for type A to cases where the elements may not be convex. 

Example 11.1. Consider the subset of permutations S n {p) avoiding p = S1S3S2S4 = [24153] as an 
embedded factor. Then, the element w = S4S1S3S5S2 = [251364] avoids p as an embedded factor, so 
w € S n {p), y et it contains p as a 1-line pattern. The string diagrams below depict how the extra string is 
added to p to get w: 



2415 3 251364 




1 2 3 4 5 6 



Let W c = U n >i W£ denote the convex elements in one of the classical families of irreducible Weyl 
groups, type A, B or D. Let W c ip) be the subset of the convex elements in W that is characterized by 
avoiding a single embedded factor pattern p. 

Let r(j>) be the rank of the Weyl group containing p, and let U ip) be the set of all convex elements 
in W r (p) that contain p as a factor, i.e. U c ip) are the convex elements in the upper order ideal generated 
by p in the two-sided weak order on W r f p y We will show that when p satisfies an additional hypothesis 
called the ideal condition, avoiding p as an embedded factor is equivalent to avoiding the elements of 
U c ip) as 1-line patterns. To carry this out, we will make frequent use of the linearized, coalesced heap 
and string diagrams on connected, convex elements. 

We say that a Coxeter embedding on W reverses orientation if the labels on the corresponding linear 
Coxeter graph are reversed under the embedding. Otherwise, we say that it preserves orientation. If w 
contains an embedded factor p under an orientation preserving Coxeter embedding then we say that w 
contains p as an orientated embedded factor. 

Proposition 11.2. If w € W contains p G W as an oriented embedded factor, then w contains an 
element ofU c ip) as a 1-line pattern. 

Proof. By Lemma 7.12, we have that Heapiw) contains a shifted copy of Heapip) as a saturated set of 
lattice points. Furthermore, we can build Heapiw) from the shifted copy of Heapip) by sequentially 
adding lattice points that are maximal or minimal with respect to the intermediate heap as follows. Sup- 
pose that the shifted copy of Heapip) occupies columns i, i + 1, . . . , k in Heapiw). Then, we begin 
with the set of strings S = + . . . , k + 1} that appear in the shifted copy of Heapip). These strings 
initially correspond to the 1-line pattern p, and we show by induction that S continues to encode a 1-line 
pattern from U p as we add minimal or maximal lattice points to the heap. 

Consider the relative order of the strings in S when we add a maximal lattice point in column j to 
the heap. If the new point crosses a pair of strings that are both in S then the new string configuration 
on S corresponds to an element in the upper order ideal U ip). If the new point crosses a pair x,y of 
strings such that at most one is contained in S then the string configuration on S is unchanged. Similarly, 
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the relative order of the strings in S corresponds to an element in U c (p) when we add a minimal lattice 
point in column j. 

Hence, at the end of this inductive construction Heap(w) contains the 1-line pattern encoded by the 
strings in S, and the element corresponding to this 1-line pattern contains p as a factor. □ 

Example 11.1 shows that the converse of Proposition 1 1.2 can fail in general. However, on the special 
patterns defined below a converse can be stated. 

Definition 11.3. We say that p G W is an ideal embedded factor pattern if for every q G W^ p ^ +1 
containing p as a 1-line pattern, we have that q contains p as an oriented embedded factor. 

Proposition 11.4. If p G W c is an ideal embedded factor pattern and w G W contains p as a 1-line 
pattern, then w contains p as an oriented embedded factor. 

Proof. Consider the case that w G W r u,\ + \. By Definition 1 1.3, we have that w contains p as an oriented 
embedded factor. By Lemma 7.12, this implies that Heap(w) contains Heap(p) as a saturated subset, 
so we can highlight an instance of Heap(p) inside Heap(w). 

Now by induction, assume the proposition holds for all convex elements in U^ =1 W^ and let w G 
W^ +l . Then if w contains p as a 1-line pattern then w contains some p' G W% that also contains p as 
a 1-line pattern. By induction, Heap{p') contains a shifted copy of Heap(p) as a saturated subset of 
lattice points, we want to show that Heap(w) must also contain a copy of Heap(p). 

The string diagram imposed on Heap(w) can be obtained from the string diagram on Heap(p') by 
adding one additional string. The additional string will add extra points to the heap at each crossing. This 
string may cut through the copy C of Heap(p), but since p is ideal, the extra points added along with 
C must also contain a shifted copy of Heap(p) as a saturated subset of lattice points by Definition 11.3. 
Therefore by Lemma 7.12, w contains p as an oriented embedded factor. □ 

Thus combining Proposition 11.2 and Proposition 1 1 .4, we have shown the following result. 

Theorem 11.5. Suppose P is the subset ofW c characterized by avoiding a finite combination of ori- 
ented convex embedded factors F and 1-line patterns G. If each of the elements in F' = IJpgF U c (p) is 
an ideal pattern, then P is characterized by avoiding the permutations in F' U G as 1-line patterns. 

Corollary 11.6. Under the hypotheses of Theorem 11.5, there is a polynomial time algorithm available 
to test an element ofW c for membership in P. 

Remark 1 1 .7. Note that the existence of a polynomial time algorithm is not evident from the embedded 
factor version of the characterization, because a typical element can have exponentially many reduced 
expressions by [Sta84]. 

Remark 11.8. Recall that the fully commutative elements in types A and B are automatically convex. 
In particular, the theorem applies to properties on [321] -avoiding permutations that are characterized by 
avoiding finitely many embedded factors. Similarly, any pattern class that includes the fully commutative 
basis elements from Theorem 7.3 together with [123] is convex in type D. 

Corollary 11.9. There exist a finite number of patterns of rank less than 9 in each Weyl group family 
of types A, B, D that characterize the Deodhar elements. Therefore, there exists an 0(n 8 ) test for the 
Deodhar condition in all finite Weyl groups of rank n. 

Proof. It is straightforward to verify that the embedded factor patterns from Theorem 8.1 are ideal using 
a computer. The corollary then follows from Theorem 1 1.5. □ 

We have verified by computer that the embedded factor patterns characterizing the Deodhar elements 
correspond to 75 type D 1-line patterns. 
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It is not known how to define a pattern system so that a finite characterization of the Deodhar con- 
dition in other Coxeter groups may be obtained, but Example 5.11 shows that something stronger than 
embedded factor containment is required in general. 

Question 11.10. Can Deodhar elements in other Coxeter groups be characterized by avoiding a finite 
number of root subsystem patterns? 

12. Toward Enumerating Deodhar Elements 

Stankova and West [SW04] found a homogeneous linear recurrence relation with constant coefficients 
that gives the number of 321 -hexagon avoiding permutations. 

Theorem 12.1. [SW04] The number c n of 321 -hexagon-avoiding permutations in S n satisfies the recur- 
rence 

c n +i = 6c n - llc n _i + 9c n _ 2 - 4c n _3 - 4c n _ 4 + c n _ 5 
for all n > 6 with initial conditions c\ = 1, C2 = 2, C3 = 5, C4 = 14, C5 = 42, cq = 132. 

Also, Vatter [Vat05] has obtained an enumeration scheme for these elements automatically using the 
WilfPlus Maple package. Do elegant enumerative formulas such as this exist for counting the Deodhar 
elements in type Dl 

Figure 5 shows the number of Deodhar elements for the finite Weyl groups as a fraction of the fully 
commutative elements. The latter were enumerated by [Ste98]. 
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Figure 5. Enumeration of Deodhar elements 



It would be interesting if one could find an enumerative formula for the number of Deodhar elements 
in type D. More generally, what can be said about enumerating families avoiding a list of embedded 
factor patterns? 

It is possible to show that in type A, the embedded factor pattern classes satisfy a Stanley-Wilf bound 
using a theorem of Tenner. 

Theorem 12.2. [Ten06b] Ifp G Sk avoids [2143] and w G S n contains pas a 1-line permutation pattern, 
then w contains p as an oriented embedded factor. 

We denote the set of elements from S n that avoid p as an embedded factor by S n (p), and the set of 
elements from S n that avoid [p] as a 1-line permutation pattern by S n \p]. 

Corollary 12.3. For all permutations p, there exists a constant c = c p such that \S n (p)\ < c n . 

Proof. We first show how to construct a permutation q G that contains p as a factor, but avoids 
[2143]. Suppose p G contains a [2143] instance. Begin by choosing the leftmost position i in the 
1-line notation for p from the set of all positions that play the role of 4 in any [2143] instance of p. Then, 
we can multiply p on the right by the adjacent transposition to move pi one position to the left. Note 
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that < pi or else we could have chosen to play the role of 4 in any [2143] instance in which 
participates, contradicting that p, was chosen to be leftmost. 

By continuing to move the entry pi to the left in a reduced fashion, we can eventually move it past 
the leftmost entry that plays the role of 1 in any [2143] instance in which pi plays the role of 4. Having 
removed all of the [2143] instances where the entries that play the role of 4 occur in positions < i, we 
choose the next leftmost position that plays the role of 4 in some [2143] instance and repeat the argument. 
The resulting element q contains p as a factor, and contains no [2143] instances. 

Hence, if w contains q as an embedded factor, then it contains p as an embedded factor, so contrapos- 
itively S n (p) C S n (q) and by Theorem 12.2 we have S n (q) C S n [q\. We can apply the Marcus-Tardos 
Theorem [MT04] to S n [q] since it is expressed as a 1-line permutation pattern class, and we obtain the 
upper bound. □ 
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